Generatable intelligent-driven full-automatic microbial colony early counting method and equipment

Abstract

The invention relates to a generative intelligent-driven full-automatic microbial colony early counting method and equipment, and belongs to the technical field of microbial experiment equipment. The invention provides a generative intelligent-driven full-automatic microbial colony early counting device which comprises a base structure, and a gradient dilution module, a point plate inoculation module, a constant-temperature culture system and an AI detection module which are mounted on the base structure, the invention aims to solve the systematic technical problems that the traditional manual operation is low in efficiency, serious in error accumulation, high in consumable cost and too long in counting time, and the existing automatic equipment is limited in experimental flux, too high in equipment cost, insufficient in experimental flexibility and the like.

Description

Technical Field

[0001] The present invention relates to a generative intelligently driven fully automatic microbial colony early counting method and equipment, belonging to the technical field of microbial experimental equipment, and in particular to an automated microbial inoculation counting technology and equipment that integrates efficient colony image enhancement and accurate prediction, gradient dilution, high-throughput inoculation point plates, early colony counting and intelligent control. Background Art

[0002] As a core technology in fields such as food safety, clinical diagnosis, and environmental monitoring, microbial colony counting testing has a direct impact on public health safety and product quality control due to its efficiency and accuracy. Traditional plate count methods, which rely on manual operations, have three inherent flaws: First, the testing cycle is as long as 24-48 hours, making it difficult to meet the needs of rapid response to sudden microbial contamination incidents; second, in terms of counting throughput, traditional methods can usually only perform a limited number of experiments on a single culture dish, failing to fully utilize the effective area of ​​the culture dish, resulting in low counting throughput and waste of experimental consumables; third, the operation process involves multiple manual steps, introducing a risk of human error of up to 15%. These bottlenecks severely restrict the application performance of microbial testing in modern quality control systems.

[0003] In recent years, with the rapid development of artificial intelligence and automation technologies, the field of microbial colony counting has undergone technological innovation. Machine vision technology can automatically identify and count colonies, and deep learning algorithms can extract complex feature information from images. However, to achieve truly rapid counts, relying solely on the identification of mature colonies is clearly insufficient. Research has shown that the early growth stages of microorganisms contain a wealth of characteristic information. Accurate identification and prediction at this stage could shorten the overall detection cycle by over 50%. However, this technological breakthrough faces significant challenges: early colonies are typically less than 200 μm in diameter, and their morphological features are less distinct; variations in culture medium composition and conditions can lead to fluctuations in image quality; and differences in growth characteristics among different bacterial species further complicate identification. Existing algorithms have low accuracy for identifying microscopic colonies in the early stages of culture, and the generalization of these models urgently needs to be improved.

[0004] In addition, although some automated equipment has been introduced into the current field of microbial testing to improve operational efficiency, these devices generally have limitations such as single functions and insufficient system integration, making it difficult to achieve full-process automation from gradient dilution of bacterial solution, precision inoculation to colony counting. In particular, there are significant shortcomings in the analysis of early colony characteristics and compression of counting cycles. For example, although the fully automatic bacterial spreading inoculator can complete standardized coating through the synergy of the rotating platform and the scraping coater, significantly reducing manual operation errors, it only supports single-concentration coating and cannot achieve gradient dilution or multivariate counting. It still needs to rely on other equipment to complete the pre-processing steps; and although the spiral inoculator achieves gradient distribution of bacterial solution through rotary coating technology, its throughput improvement is limited, and it still requires a full 24-hour culture cycle to obtain countable colony morphology, which cannot meet the needs of early counting. More importantly, the current level of modularity of existing microbial inoculation detection equipment is low, and there is a lack of effective coordination between the various functional modules, resulting in low overall operational efficiency.

[0005] Therefore, there is an urgent need to develop a microbial inoculation and counting system that can achieve full-process automated operation, high experimental accuracy and throughput, fast counting time, and low cost to meet the growing needs in the field of modern microbial counting. Summary of the Invention

[0006] In order to solve the above problems, on the first aspect, the present invention provides a generative intelligent-driven fully automatic microbial colony early counting equipment, which aims to solve the systemic technical problems of low efficiency of traditional manual operation, serious error accumulation, high cost of consumables, long counting time, and limited experimental throughput, high equipment cost, and insufficient experimental flexibility of existing automated equipment.

[0007] The present invention proposes a generative intelligent-driven fully automatic microbial colony early counting method. The full process automation of microbial colony early counting is achieved by using the AI ​​detection module of the fully automatic microbial colony early counting equipment to intelligently coordinate the gradient dilution module, the spot plate inoculation module, and the constant temperature culture system. The method includes using the gradient dilution module to control the pipette to aspirate PBS to complete the gradient dilution; controlling the three-axis motion mechanism of the gradient dilution module to drive the pipette to spot samples on the culture dish; controlling the multi-degree-of-freedom robotic arm of the spot plate inoculation module to close the culture dish and start the constant temperature system to maintain the culture environment for cultivation; and using the AI ​​detection module to regularly collect images and process and analyze them to obtain CFU counting results.

[0008] Furthermore, the fully automatic microbial colony early counting equipment includes:

[0009] The basic structure includes a shell and a bottom fixing plate arranged at the bottom of the shell;

[0010] The gradient dilution module includes a three-axis motion mechanism fixed to a bottom fixed plate, a pipette being movably connected to the three-axis motion mechanism, and the three-axis motion mechanism can control the pipette to move in three dimensions;

[0011] The plate inoculation module includes a multi-degree-of-freedom robotic arm fixed to the bottom fixed plate and located in front of the three-axis motion mechanism. The multi-degree-of-freedom robotic arm holds the culture dish and can clamp and remove the cover of the culture dish before spotting, and accurately reposition and seal it after spotting.

[0012] A constant temperature culture system is installed inside the housing and located above the gradient dilution module and the spot plate inoculation module;

[0013] The AI ​​detection module includes an imaging system arranged on a pipette, wherein the imaging system can synchronously perform radial scanning on the culture dish when the pipette moves, and obtain the CFU counting result by computing and processing the scanned image data.

[0014] Furthermore, the three-axis motion mechanism includes a bracket fixed to the bottom fixed plate. The bracket is a rectangular aluminum profile frame structure. Two X-axis linear motion modules are respectively provided along the parallel rectangular sides of the bracket. The two X-axis linear motion modules are respectively provided with X-axis sliders. The two X-axis sliders are bridged by a Y-axis linear motion module. The Y-axis linear motion module is provided with a Y-axis slider. The Y-axis slider is connected to the pipette through a pipette fixture. A Z-axis module is configured on the pipette fixture. The Z-axis module includes a ball screw arranged in the Z-axis direction. The pipette is connected to the ball screw. The AI ​​detection module is integrated into the Z-axis module.

[0015] Furthermore, the multi-degree-of-freedom robotic arm is a six-degree-of-freedom robotic arm, including a double U-shaped plate, a single U-shaped plate, a flange, a metal main steering plate, and a clamp connected in sequence. Connectors are provided between the double U-shaped plate and the single U-shaped plate, and between the metal main steering plate and the clamp. Servo gears are provided between the double U-shaped plate and the metal main steering plate and the clamp, and the clamp clamps the culture dish.

[0016] Furthermore, the clamp is a composite clamp structure formed by stacking and connecting multiple layers of clamp monoliths via connecting columns.

[0017] Furthermore, the double U-shaped plate of the multi-degree-of-freedom robotic arm is fixed to the bottom fixing plate through a robotic arm seat. The robotic arm seat includes a large base plate fixed to the bottom fixing plate and a double-through stud fixed above the large base plate. Multi-layer circular rings are installed on the double-through studs, and the multi-layer circular rings connect the double U-shaped plates.

[0018] Furthermore, the constant temperature culture system maintains uniform heating inside the equipment through a fan and a heat sink. The constant temperature culture system includes a fan, a heating plate, and a thermocouple fixed on the inside of the shell. Through the joint action of the heating plate, the fan, and the thermocouple, closed-loop control can be performed to maintain a culture environment of 37±1°C inside the shell.

[0019] Furthermore, one of the inner sides of the housing is provided with a heat sink, and an ultraviolet disinfection lamp is installed on the heat sink for air disinfection and sterilization;

[0020] Furthermore, the bottom fixing plate is also fixed with blue-cap bottles, discarded gun tip boxes, gun tip boxes, centrifuge tube racks and a plurality of other culture dishes arranged thereon.

[0021] Furthermore, the movement of the pipette is controlled by a three-axis motion mechanism, so that the imaging system performs radial scanning on the culture dish and obtains image data. The AI ​​detection module also has a built-in YOLO model and GAN network. The YOLO model can first identify the colony position, and then the GAN network can predict the colony growth trend, and finally output the CFU count result.

[0022] Furthermore, the YOLO model is an instance segmentation model of YOLOv8-Seg. The backbone network in the instance segmentation model of YOLOv8-Seg first extracts multi-level features of the image to enhance the ability to distinguish target colonies from the background; then, the feature fusion layer integrates these features to optimize the recognition effect of small target colonies; finally, the segmentation head generates accurate pixel-level segmentation results, separates colonies of different dilutions from background noise, and suppresses non-target interference factors such as scratches and bubbles on the agar surface; based on the pixel-level segmentation results, the instance segmentation model of YOLOv8-Seg is used to perform a colony quantitative analysis process, including: first, real-time counting of the detected colonies, and then automatic conversion to colony concentration (CFU / mL) according to the preset dilution factor and sample volume parameters, and finally outputting standardized quantitative detection results. The entire process realizes automated processing from image analysis to concentration calculation.

[0023] The GAN network can enhance colony image data to predict colony growth trends, including: taking 10-hour colony images as input, reconstructing colony images with 24-hour culture characteristics through the generator network G; the discriminator network uses an adversarial learning mechanism to distinguish true / false between the generated images and the real 24-hour target images, and provides feedback to optimize the generator; the generated images are compared with the target images, and the generation quality is verified through structural similarity evaluation; the generator optimizes image distribution matching and ultimately outputs enhanced data with both temporal scalability and morphological fidelity, providing 24-hour high-fidelity fungal colony images for subsequent analysis.

[0024] Beneficial effects of the present invention:

[0025] This invention uses generative AI to identify and predict colony counts, generating colony images through algorithms, achieving a 98% agreement with experimental results, significantly shortening incubation time by over 12 hours. The system implements automated control of the entire process, from gradient dilution and high-throughput spot plate inoculation to early colony detection, significantly reducing manual intervention, lowering operational errors, and improving experimental repeatability, reliability, counting throughput, and efficiency. Furthermore, through high-throughput spot plate inoculation technology, the system is capable of high-density inoculation on a single culture dish, significantly increasing experimental throughput and meeting the needs of large-scale testing. Furthermore, the introduction of an intelligent control system optimizes the experimental process, shortens experimental time, and reduces the risk of cross-contamination. The invention also reduces experimental costs and waste generation by reducing the use of disposable consumables, thus complying with environmental requirements. Overall, the system, with its high efficiency, accuracy, and low cost, addresses the challenges of traditional microbial detection methods, such as long time, low efficiency, large errors, and insufficient throughput. It provides an advanced detection solution for the food industry, environmental monitoring, and other fields, with broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a three-dimensional schematic diagram of the overall structure of a fully automatic microbial colony early counting equipment in one embodiment of the present invention.

[0027] Figure 2 This is a side view of the overall structure of a fully automatic microbial colony early counting equipment in one embodiment of the present invention.

[0028] Figure 3 This is a schematic diagram of the positions of the gradient dilution module, the spot plate inoculation module, and the constant temperature culture system in one embodiment of the present invention.

[0029] Figure 4 Schematic diagram of the structure of the gradient dilution module in one embodiment of the present invention.

[0030] Figure 5 This is a structural side view of a gradient dilution module in one embodiment of the present invention.

[0031] Figure 6 This is a structural side view of a spot plate inoculation module in one embodiment of the present invention.

[0032] Figure 7 Schematic diagram of the three-dimensional structure of a multi-degree-of-freedom robotic arm in one embodiment of the present invention.

[0033] Figure 8 This is a top view of the partial structure fixed on the base plate in one embodiment of the present invention.

[0034] Figure 9 This is a schematic structural diagram of the pipette fixing part connected to the pipette in one embodiment of the present invention.

[0035] Figure 10 Schematic diagram of 9 and 12 area spot plates in a 90 mm circular culture dish in one embodiment of the present invention.

[0036] Figure 11 The following is a comparison photo of bacteria counts by the coating method and the spot plate method in one embodiment of the present invention.

[0037] Figure 12 The figure is a comparative histogram of bacterial counts by the coating method and the spot plate method in one embodiment of the present invention.

[0038] Figure 13 This is a flow chart of fully automatic early counting of microbial colonies in one embodiment of the present invention.

[0039] Figure 14 This is a schematic diagram of the YOLOv8-Seg instance segmentation network model evaluation in one embodiment of the present invention. Taking Staphylococcus aureus and Rhamnosus rhamnosus as examples, the (B / C) confusion matrix is ​​used to quantify the network's segmentation performance for high, medium, and low concentration areas.

[0040] Figure 15 This is a schematic diagram of the evaluation of the GAN generative adversarial network model in one embodiment of the present invention. Taking Staphylococcus aureus and Rhamnosus as examples, the network's performance in colony temporal feature reconstruction is verified in multiple dimensions through pixel-level scatter regression analysis (B / E), GAN network loss curve (C / F), and PSNR-SSIM three-dimensional index (D / G).

[0041] Figure 16 This figure is a schematic diagram of the target detection function evaluation of the YOLOv8 network model in one embodiment of the present invention. Taking Staphylococcus aureus and Lactobacillus rhamnosus as examples, the performance advantages and computational efficiency of the optimized network in the real-time counting of low-concentration bacteria are verified through the dynamic curve of multi-model training accuracy (B / C).

[0042] In the figure, 1. lens; 2. fixing bolt; 3. lens fixing plate; 4. pipette fixing part; 5. pipette; 6. lead screw nut; 7. 10μl pipette tip; 8. guide rail; 9. base; 10. motor plate; 11. fixing part; 12. back plate; 13. base fixing part; 14. blue cap bottle; 15. discarded pipette tip box; 16. handle; 17. centrifuge tube; 18. 100μl pipette tip; 19. centrifuge tube rack; 20. pipette tip box; 21. culture dish; 22. single jaw; 23. servo; 24. connector; 25. single U-shaped plate; 26. double U-shaped plate; 27. multi-layer ring; 28. large bearing; 29. ​​ring; 30. motor fixing seat; 31. LED light; 32. Thermocouple; 33. Temperature controller; 34. Angle code; 35. Bracket; 36. X-axis slider; 37. Motor; 38. Plum blossom coupling; 39. Ball screw; 40. Auxiliary rail; 41. Base; 42. Heat sink; 43. Drag chain; 44. UV disinfection lamp; 45. Hinge; 46. Housing; 47. 20μl gun tip; 48. Bottom fixing plate; 49. Double-through stud; 50. Small heating plate fixing; 51. Heating plate; 52. Fan; 53. L-shaped double-track connector; 54. Flange; 55. Drag chain connecting plate; 56. Drag chain support; 57. Double-gauge connecting plate; 58. Connecting column; 59. Metal main steering wheel; 60. Large base plate. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] In the present invention, unless otherwise expressly specified or limited, the terms "connected," "connect," and "fixed" should be understood in a broad sense. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0045] In the present invention, the “upper” refers to the direction in which each component faces away from the ground, and the “lower” refers to the direction in which each component faces away from the ground.

[0046] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0047] The present invention proposes a generative intelligent-driven fully automatic microbial colony early counting method. The full process automation of microbial colony early counting is achieved by using the AI ​​detection module of the fully automatic microbial colony early counting equipment to intelligently coordinate the gradient dilution module, the spot plate inoculation module, and the constant temperature culture system. The method includes using the gradient dilution module to control the pipette to aspirate PBS to complete the gradient dilution; controlling the three-axis motion mechanism of the gradient dilution module to drive the pipette to spot samples on the culture dish; controlling the multi-degree-of-freedom robotic arm of the spot plate inoculation module to close the culture dish and start the constant temperature system to maintain the culture environment for cultivation; and using the AI ​​detection module to regularly collect images and process and analyze them to obtain CFU counting results.

[0048] Furthermore, if Figures 1-9 As shown, the fully automatic microbial colony early counting equipment includes a basic structure and a gradient dilution module, a spot plate inoculation module, a constant temperature culture system, and an AI detection module installed on the basic structure, wherein:

[0049] The basic structure includes a housing 46 and a bottom fixing plate 48 disposed at the bottom of the housing 46; Figure 9 In the example shown, a blue-capped bottle 14, a discarded tip box 15, a tip box 20, and a centrifuge tube rack 19 are fixed to the bottom fixing plate 48, along with six culture dishes. The tip box 20 stores 10μl tips 7, 100μl tips 18, and 20μl tips 47 for different needs. The centrifuge tube rack 19 is equipped with multiple centrifuge tubes 17. Furthermore, an LED light is provided on the bottom fixing plate 48 to facilitate work in dimly lit environments.

[0050] The gradient dilution module includes a three-axis motion mechanism fixed to a bottom fixed plate. Figure 1-Figure 5In the example, the three-axis motion mechanism includes a bracket 35 fixed to the bottom fixed plate 48. The bracket 35 is a rectangular aluminum profile frame structure. Two X-axis linear motion modules are respectively provided along the parallel rectangular sides of the bracket 35. The X-axis linear module is provided with a motor 37 and an X-axis module connected to the motor through a plum blossom coupling 38. One end of one of the X-axis modules is provided with an X-axis drag chain. Two X-axis sliders 36 are respectively provided on the two X-axis modules. The two X-axis sliders 36 are bridged with Y The Y-axis linear moving module is provided with a Y-axis slider and a Y-axis drag chain 43, the Y-axis slider is connected to the pipette gun 5 through the pipette gun fixing part 4, the pipette gun fixing part 4 is provided with a Z-axis module, the Z-axis module includes a ball screw 39 arranged in the Z-axis direction, the pipette gun 5 is connected to the ball screw 39, and the pipette gun 5 is driven to move in the X-axis and Y-axis directions respectively by the X-axis linear moving module and the Y-axis linear moving module, and the pipette gun 5 is driven to move in the Z-axis direction by the ball screw 39.

[0051] The plate inoculation module includes a multi-degree-of-freedom robotic arm fixed to the bottom fixed plate 48 and located in front of the three-axis motion mechanism. Figure 6-Figure 7 In the example of FIG, the multi-degree-of-freedom manipulator is a six-degree-of-freedom manipulator, comprising a double U-shaped plate 26, a single U-shaped plate 25, a flange 54, a metal main steering plate 59, and a clamping claw connected in sequence. Connectors 24 are provided between the double U-shaped plate 26 and the single U-shaped plate 25, and between the metal main steering plate 59 and the clamping claw. A servo 23 is provided between the double U-shaped plate 26 and the metal main steering plate 59 and the clamping claw. The clamping claw is used to clamp the culture dish 21. The multi-degree-of-freedom manipulator can clamp and hold the culture dish 21 before spotting. The lid of the culture dish 21 is removed and, after sample application, accurately reset and sealed. The dual U-shaped plates 26 of the multi-DOF manipulator are secured to the bottom fixing plate via a manipulator base. The manipulator base includes a large base plate 60 secured to the bottom fixing plate and a double-through stud 49 secured above the large base plate 60. The double-through stud 49 is mounted with a multi-layered ring 27, which connects the dual U-shaped plates 26, enhancing the stability of the multi-DOF manipulator and preventing vibration interference from other surrounding components during operation. Preferably, the gripper is a composite gripper structure formed by stacking multiple single-layered grippers 22 connected by connecting posts 58, facilitating precise gripping of the culture dish 21 while improving accuracy.

[0052] A constant temperature culture system is installed on the outside of the shell 46; the constant temperature culture system uses a fan and a heat sink to ensure temperature uniformity; the constant temperature culture system includes a fan 52, a heating plate 51, and a thermocouple 32 fixed to the inside of the shell 46. Through the combined action of the heating plate 51, the fan 52, and the thermocouple 32, a closed-loop control can be performed to maintain a culture environment of 37±1°C in the shell.

[0053] The AI ​​detection module is integrated on the Z-axis module and includes an imaging system arranged on the pipette. The imaging system includes a lens fixing plate 3 fixed to the pipette 5 and a lens 1 arranged on the lens fixing plate. The lens 1 can synchronously perform radial scanning on the culture dish 21 when the pipette 5 moves, and obtain the CFU counting result by computing the scanned image data.

[0054] Furthermore, the movement of the pipette 5 is controlled by a three-axis motion mechanism, so that the imaging system performs radial scanning on the culture dish 21 and obtains image data. The AI ​​detection module also has a built-in YOLO model and a GAN network. The YOLO model can first identify the colony position, and then predict the colony growth trend through the GAN network, and finally output the CFU count result.

[0055] Furthermore, a heat sink 42 is provided inside the device, and an ultraviolet disinfection lamp 44 is installed on the heat sink 42 for air disinfection and sterilization before the experiment; a temperature controller 33 is provided outside the housing 46 to facilitate temperature control by the operator.

[0056] Furthermore, the YOLO model is an instance segmentation model of YOLOv8-Seg. The backbone network in the instance segmentation model of YOLOv8-Seg first extracts multi-level features of the image to enhance the ability to distinguish target colonies from the background; then, the feature fusion layer integrates these features to optimize the recognition effect of small target colonies; finally, the segmentation head generates accurate pixel-level segmentation results, separates colonies of different dilutions from background noise, and suppresses non-target interference factors such as scratches and bubbles on the agar surface; based on the pixel-level segmentation results, the instance segmentation model of YOLOv8-Seg is used to perform a colony quantitative analysis process, including: first, real-time counting of the detected colonies, and then automatic conversion to colony concentration (CFU / mL) according to the preset dilution factor and sample volume parameters, and finally outputting standardized quantitative detection results. The entire process realizes automated processing from image analysis to concentration calculation.

[0057] It should be understood that YOLOv8, as the mainstream iterative model in the YOLO series, is a multi-task learning framework that supports tasks such as target detection, instance segmentation, and pose estimation. The target detection function achieves object positioning and classification through bounding box regression; the instance segmentation function further implements pixel-level instance-aware segmentation based on target detection. The two are parallel branches of different functions within the same framework. In the entire colony quantitative analysis process, the instance segmentation function is used to extract complete low-concentration colony masks from multi-concentration colonies. Its pixel-level segmentation accuracy can effectively separate adjacent colonies and accurately extract the spatial outlines of low-concentration colonies. The target detection function extracts multi-scale features through convolutional neural networks and uses an anchor-free mechanism to achieve bounding box regression to complete the positioning and counting of individual colonies.

[0058] The GAN network is a generative adversarial network (GAN), a deep learning model that generates realistic data through adversarial training. Its core idea is to achieve data generation through the dynamic game of two neural networks. It can enhance colony image data to predict colony growth trends, including: taking 10-hour colony images as input, the generator network G reconstructs colony images with 24-hour culture characteristics; the discriminator network uses an adversarial learning mechanism to distinguish true / false between the generated images and the real 24-hour target images, and provides feedback to optimize the generator; the generated images are compared with the target images, and the generation quality is verified by structural similarity evaluation; the generator optimizes image distribution matching, and ultimately outputs enhanced data with both temporal scalability and morphological fidelity, providing 24-hour high-fidelity fungal colony images for subsequent analysis.

[0059] Example 1

[0060] Taking Staphylococcus aureus and Rhamnosus bacillus as examples, the YOLOv8-Seg instance segmentation model is used to verify the effect of suppressing background noise interference in non-target areas. Figure 14 shown.

[0061] Figure 14 Figure A shows the entire process of the YOLOv8-Seg instance segmentation network processing microbial images: the backbone network (Backbone) first extracts multi-level features of the image to enhance the ability to distinguish target colonies (such as Staphylococcus aureus and Rhamnosus bacillus) from the background; the feature fusion layer (Neck) integrates these features to optimize the recognition effect of small target colonies; the segmentation head (Head) generates accurate pixel-level segmentation results, effectively separating colonies of different dilutions from background noise, while significantly suppressing non-target interference factors such as scratches and bubbles on the agar surface.

[0062] Figure 14The 3×3 confusion matrix presented by BC quantifies the segmentation performance of the YOLOv8-Seg network for Staphylococcus aureus and Bacillus rhamnosus in high, medium, and low concentration regions. The matrix uses a blue color scale (0.0-0.8) to visually display the distribution of classification proportions, with overall accuracy reaching 96.99% and 97.26%, respectively. Recognition performance for S. aureus was: High (98%), Middle (85%), Low (91%); while for Bacillus rhamnosus, it achieved: High (97%), Middle (95%), Low (94%). The data shows that the model demonstrates excellent segmentation performance (>97%) in high-concentration regions for both bacteria, but there are differences in recognition performance in medium-concentration regions: Bacillus rhamnosus (95%) shows significantly better discrimination stability than S. aureus (85%). Notably, there was a 10% misclassification shift towards the low-concentration category for medium-concentration S. aureus samples. The model's discriminative performance for the High and Low categories is significantly better than that for the Middle category. Misclassification of the Middle category primarily manifests as a shift toward the Low category, with a misclassification rate of 10%. Through structured numerical layout and color contrast, the YOLOv8-Seg network can effectively segment low-density areas, providing a high-quality dataset for subsequent GAN networks.

[0063] Example 2

[0064] Taking Staphylococcus aureus and Rhamnosus as examples, the effect of generative adversarial network (GAN) in generating time series images of microbial colonies was verified. Figure 15 shown.

[0065] Figure 15 A demonstrates the colony image data augmentation process based on a generative adversarial network (GAN). First, a 10-hour colony image is input, and the generator network G reconstructs a colony image with 24-hour culture characteristics. The discriminator network uses an adversarial learning mechanism to distinguish true from false between the generated image and a real 24-hour target image, providing feedback to optimize the generator. The generated image is compared with the target image, and the quality of the generated image is verified through structural similarity evaluation. The generator optimizes image distribution matching and ultimately outputs augmented data that combines temporal scalability and morphological fidelity, providing high-fidelity 24-hour colony images for subsequent analysis.

[0066] Figure 15 BD presents comprehensive evaluation results of the performance of generative adversarial networks (GANs) for image generation of Staphylococcus aureus and Rhamnosus bacilli. Figure 15In Figures B and 15E), the reconstruction errors of the generated images of Staphylococcus aureus and Rhamnosus rhamnosus are both less than 0.15 units, with only slight overexposure in the range of 19.8-20.2 and 19.8-20.0, respectively, proving that the GAN network can achieve high-fidelity reconstruction of the morphological characteristics of both types of colonies. Training dynamic analysis ( Figure 15 C and Figure 15 F) shows that the validation loss of both strains converged stably to 0.05 after 50 training cycles, and the validation loss was always 15% lower than the training loss, indicating that the network has excellent generalization ability for both types of bacteria. Figure 15 D and Figure 15 G) The system compared the generation quality of Staphylococcus aureus and Bacillus rhamnosus by the GAN network. The results showed that Staphylococcus aureus achieved the best performance at Time = 10h (PSNR = 28.6dB, SSIM = 0.81), while Bacillus rhamnosus showed a peak at Time = 12h (PSNR = 30.1dB, SSIM = 0.8). The PSNR and SSIM indicators of the two types of colonies showed a synchronous improvement trend with the extension of culture time. Among them, Staphylococcus aureus has a faster growth rate and can accumulate sufficient biomass features in a shorter time; while Bacillus rhamnosus can eventually achieve comparable image quality, but it is relatively slow and requires an additional 2 hours of culture to provide equivalent feature information. This result verifies the optimization value of the species-specific time threshold. This discovery provides a precise basis for the regulation of time nodes for the generation of time-series images of different microorganisms.

[0067] Example 3

[0068] Figure 16 A demonstrates the colony quantification analysis process based on the YOLOv8 object detection network: Using 24-hour colony images as input, the Backbone network extracts multi-layer features. The Neck feature fusion layer integrates colony feature information at different scales, and Head detection ultimately locates and classifies the colonies. The system counts detected colonies in real time and automatically converts them to colony concentration (CFU / mL) based on preset dilution factors and sample volume parameters, outputting standardized quantitative test results.

[0069] In the target counting phase, Figure 16The dynamic accuracy curves of the BC multi-model training process show that the YOLOv8s network exhibits excellent detection performance for both bacterial colonies: for Staphylococcus aureus, YOLOv8s achieved 98% detection accuracy after 200 training cycles, a 2.3% improvement over YOLOv5s, significantly outperforming DETR (91%) and Faster R-CNN (87%). For Bacillus rhamnosus, YOLOv8s achieved 98% detection accuracy after 200 training cycles, a 2.9% improvement over YOLOv5s, significantly outperforming DETR (94%) and Faster R-CNN (89%). This dynamic multi-model comparison reveals key differences in object detection algorithms in terms of representation learning efficiency and convergence stability. Through comparative analysis, the YOLOv8s network was selected as the final object detection model.

[0070] Example 4

[0071] The gradient dilution module uses sterile PBS to supply the pipette 5 with precise dilution of the bacterial solution. Sterile PBS is dynamically supplied from a 100mL sterile volumetric flask. The pipette 5 can be equipped with a 100-1000μL pipette tip. According to the preset program, 900μL of sterile PBS is dispensed into the EP tube. Then, 100μL of the initial bacterial solution is transferred to PBS through the pipette tip for step-by-step dilution. After each level of dilution, 10 pipetting and mixing operations are automatically performed to ensure the uniformity of the bacterial solution concentration. The entire process realizes the automatic switching of different specifications of pipette tips through the pipette tip box replacement operation. Waste pipette tips are placed in the waste pipette tip box 15 and collected by the biosafety sealing device. The capacity is 200, and the positioning error does not exceed ±0.3mm.

[0072] In this embodiment, the plate inoculation module is combined with a three-axis motion mechanism drive device to complete the high-density sample spotting operation. The three-axis motion mechanism has a repeatability accuracy of ±0.05mm and a motion speed of up to 240mm / s. It drives the pipette 5 equipped with a 1-10μL tip to place the sample on the culture dish as shown in the following figure. Figure 10 Inoculation is performed in a circular sector layout with 9 zones (equally divided by 40°) or 12 zones (equally divided by 30°). Ten 1μL ± 5% inoculation points are evenly spaced radially in each zone, with a radial spacing of 3mm ± 0.2mm and a circumferential angular error of ≤ ± 0.5°. The tip can be replaced with an ejection-type mechanism. During inoculation, the ejection-type mechanism automatically detaches the tip by applying a vertical downward force of ≥ 8N. Tip replacement is then performed automatically after a single batch of culture dishes has been inoculated.

[0073] Example 5

[0074] like Figure 11-12As shown, by comparing the performance of the traditional coating method and the spot plate inoculation method of the present invention in colony counting, the experimental results showed that there was no significant difference in colony counting results between the two methods (p>0.05), indicating that the spot plate method has significant operational advantages while ensuring data accuracy. Specifically, the spot plate method uses a high-density sector-shaped spotting design (9 or 12 zones × 10 spots / dish) to increase the single-dish experimental throughput to 9-12 times that of the traditional coating method, and avoids the use of consumables such as coating rods and a large number of plates, reducing consumable costs by approximately 90%. Figure 11 As can be seen from the figure, the colony distribution formed by the spot plate method of the present invention shows obvious regularity ( Figure 11 Right), compared with the random distribution of colonies produced by the traditional coating method ( Figure 11 This regular distribution is achieved thanks to precise spot positioning (radial spacing 3mm±0.2mm, circumferential angle error ≤±0.5°), which enables each inoculation point to form an independent colony growth area. The matching high-resolution industrial camera lens and AI detection module enable the rapid and accurate identification and counting of colonies in each area. Figure 12 Quantitative comparison data further confirmed that, at the same initial bacterial concentration, the relative error in CFU counts between the spot plate method and the spread plate method was consistently within ±3%, with no significant difference, meeting the precision requirements for quantitative microbial testing. Through standardized spotting patterns and high-throughput design, breakthroughs in both experimental efficiency and economic benefits are possible.

[0075] Example 6

[0076] In this embodiment, the constant temperature incubation system utilizes a fan 52, a heater 51, and a thermocouple 32 to maintain a closed-loop control environment within the enclosure at 37±1°C. The fan 52 incorporates the heater 51, which, combined with air circulation, ensures uniform temperature distribution. A temperature sensor provides real-time feedback for rapid temperature adjustments.

[0077] In this embodiment, the intelligent control system integrates the functions of each module, supports the customization of 9-zone or 12-zone spotting modes, and coordinates the linkage operation of the three-axis motion mechanism, the pipette gun, and the multi-degree-of-freedom robotic arm. Within 30 seconds of powering on the system, the tip seal test, laser sensor calibration, and module zero self-test are completed. The full process automation control includes gradient dilution of bacterial solution, high-density spotting, tip replacement, culture dish flipping, and AI detection. A single batch of 6 culture dishes takes less than 10 minutes, shortening the detection cycle by more than 12 hours compared to traditional methods.

[0078] This inoculation device innovatively integrates a gradient dilution module, a spot plate inoculation module, and an intelligent AI detection module. Through an integrated workflow, it realizes sample gradient dilution, precise inoculation of microbial culture solutions, and early intelligent identification of colonies, with unmanned operation throughout the process. Based on the dynamic path planning and actuator coordination mechanism of the intelligent control system, the device has made a breakthrough in solving the technical bottlenecks of low manual operation efficiency and high error rate in traditional microbial testing. The experimental throughput has been increased to more than 9 times that of traditional methods, and the single-batch operation time has been shortened by 90%. Its core technologies lie in the high-precision motion control of the three-axis motion mechanism (±0.05mm positioning accuracy), the automatic replacement system for multiple specifications of gun tips (covering the full range of 1-1000μL), and the advanced prediction capabilities of the generative AI algorithm, which enables the accurate prediction of the 24-hour colony growth status within a 10-hour culture period, greatly improving the timeliness of detection.

[0079] Although the present invention has been disclosed above in terms of preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the claims.

Claims

1. A generative intelligent driven fully automatic microbial colony early counting method, characterized in that: By using the AI ​​detection module of the fully automatic microbial colony early counting equipment to intelligently coordinate the gradient dilution module, the spot plate inoculation module, and the constant temperature culture system, the full process of microbial colony early counting is automated, including: The gradient dilution module is used to control the pipette to aspirate PBS to complete the gradient dilution. The three-axis motion mechanism of the gradient dilution module is controlled to drive the pipette to spot samples on the culture dish. The multi-degree-of-freedom robotic arm of the spot plate inoculation module is controlled to close the culture dish and start the constant temperature system to maintain the culture environment for cultivation. The AI ​​detection module is used to regularly collect images and process and analyze them to obtain the CFU count results.

2. The method according to claim 1, characterized in that The fully automatic microbial colony early counting equipment includes: The basic structure includes a shell and a bottom fixing plate arranged at the bottom of the shell; The gradient dilution module includes a three-axis motion mechanism fixed to a bottom fixed plate, a pipette being movably connected to the three-axis motion mechanism, and the three-axis motion mechanism can control the pipette to move in three dimensions; The plate inoculation module includes a multi-degree-of-freedom robotic arm fixed to the bottom fixed plate and located in front of the three-axis motion mechanism. The multi-degree-of-freedom robotic arm holds the culture dish and can clamp and remove the cover of the culture dish before spotting, and accurately reposition and seal it after spotting. A constant temperature culture system is installed inside the housing and located above the gradient dilution module and the spot plate inoculation module; The AI ​​detection module includes an imaging system arranged on a pipette, wherein the imaging system can synchronously perform radial scanning on the culture dish when the pipette moves, and obtain the CFU counting result by computing and processing the scanned image data.

3. The method according to claim 2, characterized in that The three-axis motion mechanism includes a bracket fixed to a bottom fixed plate, and the bracket is a rectangular aluminum profile frame structure. Two X-axis linear motion modules are respectively provided along the parallel rectangular sides of the bracket, and the two X-axis linear motion modules are respectively provided with X-axis sliders. The two X-axis sliders are bridged with a Y-axis linear motion module, and the Y-axis linear motion module is provided with a Y-axis slider. The Y-axis slider is connected to the pipette through a pipette fixing piece, and a Z-axis module is configured on the pipette fixing piece. The Z-axis module includes a ball screw arranged in the Z-axis direction, the pipette is connected to the ball screw, and the AI ​​detection module is integrated in the Z-axis module.

4. The method according to claim 3, characterized in that The multi-degree-of-freedom robotic arm is a six-degree-of-freedom robotic arm, including a double U-shaped plate, a single U-shaped plate, a flange, a metal main steering plate, and a clamping claw connected in sequence. Connectors are provided between the double U-shaped plate and the single U-shaped plate, and between the metal main steering plate and the clamping claw. Servo gears are provided between the double U-shaped plate and the metal main steering plate and the clamping claw. The clamping claw clamps the culture dish.

5. The method according to claim 4, characterized in that The clamp is a composite clamp structure formed by stacking and connecting multiple clamp monoliths via connecting columns.

6. The method according to claim 5, characterized in that The double U-shaped plate of the multi-degree-of-freedom robotic arm is fixed to the bottom fixing plate through a robotic arm seat. The robotic arm seat includes a large base plate fixed to the bottom fixing plate and a double-through stud fixed above the large base plate. Multi-layer circular rings are installed on the double-through studs, and the multi-layer circular rings connect the double U-shaped plates.

7. The method according to claim 2, characterized in that The constant temperature culture system uses a fan and a heat sink to keep the interior of the equipment evenly heated. The constant temperature culture system includes a fan, a heating plate, and a thermocouple fixed to the inside of the shell. The combined action of the heating plate, the fan, and the thermocouple can achieve closed-loop control to maintain a culture environment of 37±1°C inside the shell.

8. The fully automatic microbial colony early counting equipment according to claim 2, characterized in that: A heat sink is provided on one of the inner sides of the housing, and an ultraviolet disinfection lamp is installed on the heat sink for air disinfection and sterilization; The bottom fixing plate is also fixed with blue-cap bottles, discarded gun tip boxes, gun tip boxes, centrifuge tube racks and a plurality of other culture dishes.

9. The fully automatic microbial colony early counting equipment according to claim 2, characterized in that: The movement of the pipette is controlled by a three-axis motion mechanism, allowing the imaging system to radially scan the culture dish and obtain image data. The AI ​​detection module also has a built-in YOLO model and GAN network. The YOLO model can first identify the location of the colony, and then the GAN network can predict the colony growth trend, and finally output the CFU count result.

10. The fully automatic microbial colony early counting equipment according to claim 9, characterized in that: The YOLO model is an instance segmentation model of YOLOv8-Seg. The backbone network in the instance segmentation model of YOLOv8-Seg first extracts multi-level features of the image to enhance the ability to distinguish target colonies from the background; Subsequently, the feature fusion layer integrates these features to optimize the recognition of small target colonies. Finally, the segmentation head generates accurate pixel-level segmentation results, separating colonies of different dilutions from background noise while suppressing non-target interference factors. Based on the pixel-level segmentation results, the YOLOv8-Seg instance segmentation model was used to perform a colony quantitative analysis process, including: first, real-time counting of the detected colonies, then automatic conversion to colony concentration (CFU / mL) based on the preset dilution factor and sample volume parameters, and finally output of standardized quantitative detection results. The entire process realizes automated processing from image analysis to concentration calculation. The GAN network can enhance colony image data to predict colony growth trends, including: taking 10-hour colony images as input, reconstructing colony images with 24-hour culture characteristics through the generator network G; the discriminator network uses an adversarial learning mechanism to distinguish true / false between the generated images and the real 24-hour target images, and provides feedback to optimize the generator; the generated images are compared with the target images, and the generation quality is verified through structural similarity evaluation; the generator optimizes image distribution matching and ultimately outputs enhanced data with both temporal scalability and morphological fidelity, providing 24-hour high-fidelity fungal colony images for subsequent analysis.

Images