Microbial culture appliance image automatic classification method

Abstract

The invention provides a microbial culture appliance image automatic classification method, which comprises the following steps of: firstly, classifying a to-be-detected culture appliance image by adopting a coarse classification model, if a large classification result shows that the current culture appliance needs to be finely classified, detecting and intercepting an ROI image by adopting an ROI positioning engine, and if the ROI image is not finely classified, carrying out classification on the current culture appliance; inputting the ROI image into a fine classification model for fine classification; if the classification confidence coefficient is smaller than a set threshold value, an ROI image is detected and intercepted through an ROI positioning engine, the ROI image is input into a rough classification model for verification classification, and then a three-stage pyramid type mixed intelligent classification framework is adopted to achieve automatic and accurate classification of the microbial culture apparatus.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an automatic classification method for images of microbial culture equipment. Background Technology

[0002] In the fields of pharmaceuticals, medicine, food, and environmental monitoring, qualitative and quantitative analysis of microorganisms in products or the environment is crucial. Among them, colony culture method is the "gold standard" for microbial detection. Its core process is to inoculate the sample to be tested onto a solid culture medium and culture it under suitable conditions such as constant temperature and humidity, so that individual microbial cells proliferate to form visible colonies. Then, by observing the morphology and counting the number of colonies, the microbial status in the sample can be accurately assessed.

[0003] However, in microbiology laboratories, tasks such as colony counting, antimicrobial analysis, and environmental monitoring require the selection of different types of culture equipment based on sample characteristics (e.g., bacterial concentration, matrix composition), testing objectives (e.g., rapid screening, accurate counting), and industry standard requirements. These culture equipment mainly include, but are not limited to:

[0004] (1) Ordinary round petri dishes: These are the most widely used basic tools. Common sizes include 35mm, 55mm, 60mm, 90mm, and 150mm. They are suitable for pouring or spreading methods of culture. (2) Ordinary square petri dish: It adopts a rectangular structure design, which has a larger effective culture surface area, and is especially suitable for special experiments such as long-term colony streak isolation (such as the four-zone streak method) and strain purification; (3) Filter membrane culture dish: Used in conjunction with a membrane filtration system, it is often used to detect low bacterial count samples such as large volume water samples. The filter membrane that has trapped microorganisms is transferred to the agar surface of a regular culture dish. Filter membranes are usually divided into two types: non-grid and gridded.

[0005] (4) Contact dish: It is a special type of petri dish with agar medium surface protruding from the edge of the dish to form a regular hemisphere. It is used for microbial sampling on flat carriers such as equipment surfaces, work clothes, and laboratory benchtops to achieve direct detection of environmental microbial pollution.

[0006] (5) Test strip: Card-type pre-made culture medium is a card-type pre-made culture medium encapsulated by two layers of thin film. It has advantages such as unique color background, flat appearance and convenient use, and can realize rapid detection and preliminary screening of microorganisms.

[0007] Currently, mainstream automated colony counters or image analysis systems heavily rely on manual operation to set analysis parameters before colony identification and counting. Operators must first manually select the type of culture vessel corresponding to the current image in the software interface. Because different types of culture vessels vary significantly in size, shape, color, edge features, and background texture, this method has the following main problems and drawbacks: 1. Inefficiency: Manual selection of steps increases the complexity of the operation process, disrupts the continuity of automated analysis, and cannot meet the rapid processing needs of high-throughput samples (such as batch testing in food processing plants and multi-site samples in environmental monitoring stations), thus restricting the improvement of laboratory testing efficiency.

[0008] 2. Risk of Human Error: During long working hours, operators are prone to fatigue, negligence, or misjudging the type of equipment, leading to incorrect equipment selection. This error can directly cause deviations in the application of subsequent image preprocessing algorithms (such as edge detection thresholds and counting region delineation), resulting in inaccurate colony counting and morphological recognition errors. For example, if a contact dish is misidentified as a regular round culture dish, the system cannot recognize its convex edge features, leading to inaccurate counting region positioning and ultimately outputting incorrect detection results.

[0009] 3. Insufficient system intelligence: Existing equipment lacks the ability to adaptively recognize input images and cannot automatically match and adapt algorithms according to the inherent characteristics of culture equipment. This limits the core potential of the instrument as an "intelligent detection device," making it difficult to integrate into fully automated laboratory production line systems and hindering the digital and intelligent upgrading process in the field of microbial detection. Summary of the Invention

[0010] This application provides an automatic classification method for images of microbial culture equipment. Addressing the shortcomings of existing technologies, it offers a method that can automatically, quickly, and accurately classify images of microbial culture equipment. This eliminates manual setup steps, fundamentally avoiding analytical biases caused by human selection errors, and significantly improving the intelligence level, analytical accuracy, and overall efficiency of colony counting systems. It lays the foundation for the full automation and digitization of laboratories.

[0011] In a first aspect, embodiments of this application provide an automatic classification method for images of microbial culture equipment, comprising the following steps: S1: Input the image of the culture vessel to be detected into the pre-trained coarse classification model to obtain the major classification result and classification confidence. If the major classification result shows that the current culture vessel needs to be further classified, then proceed to step S2; if the classification confidence is less than the set threshold, then proceed to step S3. S2: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region. Extract the ROI image from the image of the culture vessel to be detected based on the ROI region. Input the ROI image into the pre-trained fine classification model and output the subclass classification result. S3: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region. Extract the ROI image from the image of the culture vessel to be detected based on the ROI region. Input the ROI image back into the pre-trained coarse classification model and output the verification classification result.

[0012] Secondly, embodiments of this application provide a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including an automatic classification method based on the image of the microbial culture vessel.

[0013] The main contributions and innovations of this invention are as follows: 1. Breakthrough accuracy: Through a three-level pyramid decision-making mechanism, especially for in-depth analysis of difficult samples, the classification error rate is effectively reduced to less than one in a thousand, achieving an accuracy of ≥99.5%.

[0014] 2. Improved detection speed: Most "clear samples" are processed quickly at the first level, and only a few difficult samples require special model adjudication calculations, resulting in an extremely short average system response time, meeting the needs of online real-time processing.

[0015] 3. Excellent robustness: It has a strong ability to adapt to challenging scenes such as blurred images, partial occlusion, different lighting, uneven lighting, and complex backgrounds.

[0016] 4. Strong scalability: When a new subclass is needed, simply train a new, small, dedicated classifier and add it to the decision chain; there is no need to retrain the entire complex model.

[0017] 5. Fully automated: It achieves "zero manual" identification of culture equipment types, which is a key step towards a fully automated microbial testing laboratory and can significantly improve the automation level and accuracy of the product.

[0018] 6. Seamless integration: This invention can be easily embedded into various existing colony counters, intelligent incubators, or image analysis software as a standalone software module or hardware chip.

[0019] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is an automatic classification method for images of microbial culture equipment according to embodiments of this application.

[0021] Figure 2 This is a schematic diagram of microbial culture equipment for ordinary round dishes, ordinary square dishes, filter membrane plates, and abnormal samples.

[0022] Figure 3 This is a schematic diagram of the test piece.

[0023] Figure 4 This is a schematic diagram illustrating the classification process of microbial culture equipment.

[0024] Figure 5 This is a schematic diagram illustrating the classification process of another type of microbial culture equipment.

[0025] Figure 6 This is a schematic diagram of the test piece classification process.

[0026] Figure 7 This is a flowchart for ROI location.

[0027] Figure 8 This is a classification logic diagram for a fine-classification model.

[0028] Figure 9 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0030] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0031] Example 1 This solution provides an automatic classification method for images of microbial culture equipment. It adopts a "three-level pyramid" hybrid intelligent classification architecture, which achieves a classification mechanism of "active observation and focused magnification" from coarse to fine through step-by-step positioning, magnification and adjudication. It creatively integrates the abstract semantic understanding ability of CNN convolutional neural network model with the precise low-level feature analysis ability of traditional algorithm. It provides an automatic classification method for microbial culture equipment with a classification accuracy of over 99.5%, fast inference speed, fine-grained classification ability (not only can it classify major categories, but it can also identify subcategories), and extremely robustness to complex samples (such as partial occlusion, non-standard lighting, presence of condensation, and presence of bacterial colony background).

[0032] like Figure 1 As shown, specifically, the automatic classification method for images of microbial culture equipment provided in this solution includes the following steps: S1: Input the image of the culture vessel to be detected into the pre-trained coarse classification model to obtain the major classification result and classification confidence. If the major classification result shows that the current culture vessel needs to be further classified, then proceed to step S2; if the classification confidence is less than the set threshold, then proceed to step S3. S2: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region, extract the ROI image from the image of the culture vessel to be detected based on the ROI region, and input the ROI image into the pre-trained fine classification model to output the subclass classification result. S3: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region. Extract the ROI image from the image of the culture vessel to be detected based on the ROI region. Input the ROI image back into the pre-trained coarse classification model to output the verification classification result.

[0033] As mentioned earlier, this solution employs a three-level pyramid decision-making mechanism for culture vessel images of different qualities or types. First, a coarse classification model is used to classify the culture vessels into major categories, obtaining the major category classification results and classification confidence. If the major category needs to be further subdivided, the ROI image is captured by the ROI localization engine and input into the sub-classification model to output the sub-class results. If the classification confidence is lower than a set threshold, the image is captured again after ROI localization and input into the coarse classification model to obtain the verification classification results. This method achieves a highly efficient balance between classification accuracy, processing efficiency, and scene adaptability through the progressive logic of "coarse classification screening - precise localization - subdivision / verification decision".

[0034] Specifically, in step S1: The "image of the culture vessel to be detected" in this solution can be a digital image transmitted from image acquisition devices such as cameras, scanners, microscopes, industrial cameras, CCD image sensors, CMOS image acquisition modules, and intelligent incubators with built-in imaging units. Regarding the "coarse classification model" mentioned in step S1, the "coarse classification model" in this scheme is trained to output the major category classification results and classification confidence of the culture vessel images. Specifically, images of various culture vessels containing the major category classification results of the labeled culture vessels are used as the training dataset, and the coarse classification model is trained using the training dataset.

[0035] Furthermore, this approach collects image sets of various culture equipment with labeled major classification results. To improve the robustness of the coarse classification model, images need to be collected under different lighting conditions, angles, backgrounds, and equipment usage states (empty dishes, dishes with colonies), and each image is labeled according to a "clear major category" to form a training dataset.

[0036] In some embodiments, the collected training dataset is image-normalized before being input into the coarse classification model for training.

[0037] Furthermore, image standardization processing methods include, but are not limited to, size normalization, color space conversion (RGB to grayscale or retaining RGB), HSV enhancement, translation, flipping, perspective, and noise filtering.

[0038] It should be noted that the images of the culture equipment to be detected collected in step S1 can also undergo the same image standardization process before being input into the pre-trained coarse classification model.

[0039] In some embodiments, the coarse classification model employs a lightweight CNN convolutional neural network model. Specifically, the coarse classification model includes a feature extraction module, fully connected layers, and a classifier. The feature extraction module includes multiple convolutional and pooling layers to automatically learn and extract deep abstract features of the input image. The classifier uses softmax to output the broad class classification result of the image or to verify the classification result and classification confidence. As mentioned earlier, this scheme uses a large training dataset to train the aforementioned CNN convolutional neural network model to obtain a coarse classification model that meets the classification requirements of this scheme.

[0040] Furthermore, the image of the culture vessel to be detected is input into a pre-trained coarse classification model. The coarse classification model will output the corresponding major category classification result and classification confidence for the image of the culture vessel to be detected, as shown below:

[0041] in, This represents the set of all types of culture equipment. This indicates the total number of categories in the major category. This represents the i-th major category. Represents a coarse classification model Category confidence level This indicates the probability output.

[0042] In embodiments of this solution, such as Figure 2 and Figure 3 As shown, Figure 2 These are schematic diagrams of ordinary round dishes, ordinary square dishes, filter membrane petri dishes, and abnormal samples. Figure 3 This is a schematic diagram of the test piece. The major classification results and verification classification results of this scheme include any one of the following: ordinary round dishes, ordinary square dishes, filter membrane plates, test pieces, and abnormal samples. Of course, the specific major classification results can be adjusted according to actual classification needs.

[0043] It should be noted that step S1 of this scheme determines whether subsequent steps need to be performed and which steps to perform based on the broad category classification results and classification confidence levels. That is, this scheme distinguishes the decision category of the culture vessel image to be detected based on the broad category classification results and classification confidence levels. If the broad category classification results indicate that the current culture vessel requires further sub-classification, then the current culture vessel image to be detected belongs to the sub-classification type; if the classification confidence level is less than a set threshold, then the current culture vessel image to be detected belongs to the verification type; if the broad category classification results do not indicate that the current culture vessel requires further sub-classification and the classification confidence level is not less than the set threshold, then the current culture vessel image to be detected belongs to the direct pass type.

[0044] Furthermore, based on the general classification results, it is first determined whether the current culture equipment needs to be further classified. If it does not need to be further classified, the classification confidence is then determined to see if it is less than a set threshold.

[0045] In some embodiments, a predefined subcategories list is provided, which records the major categories that need to be subcategorized. If the major category classification result of the culture vessel image to be detected belongs to the major category recorded in the subcategories list, then the culture vessel image to be detected belongs to the subcategories type and step S2 needs to be executed.

[0046] In this embodiment of the solution, the main categories requiring further subcategories are recorded in the subcategories list as ordinary round dishes, filter membrane plates, and test plates. Of course, the specific contents of the subcategories list can be flexibly adjusted according to requirements.

[0047] Furthermore, users can first set up a detailed category list T according to their needs, as shown below:

[0048] in, This represents a list of subcategories within a larger set of categories that need to be further subdivided. Indicates the number of undetermined types, and , This indicates the total number of categories in the major category. The set D representing the j-th undetermined type, which therefore does not require further subcategorization, is represented as:

[0049] Where S is the set of all types of culture equipment.

[0050] Furthermore, the set V of the types to be verified that need to be verified is represented as follows: ; in To set a threshold, Represents a coarse classification model Category The confidence level.

[0051] The set P of directly output types that can be directly output is represented as: .

[0052] In step S2: As mentioned above, step S2 of this scheme is to further classify the subcategories that need to be further classified. In order to improve the accuracy of the classification, the ROI positioning engine is first used to locate the ROI of the cultivation equipment, and then the classification model is used to perform further classification to output the subclass classification results.

[0053] In some embodiments, the ROI localization engine pre-sets ROI detection algorithms corresponding to different major categories of culture equipment, and matches the corresponding major category of ROI detection algorithm in the ROI localization engine according to the major category classification result of the culture equipment image to be detected.

[0054] Specifically, the ROI localization engine of this solution includes preset ROI detection algorithms such as ordinary plate ROI detection algorithm, filter membrane ROI detection algorithm, and test piece localization algorithm. The ROI detection algorithms include traditional algorithm localization and CNN region of interest prediction. Different ROI detection algorithms can be adapted to specific conditions based on the characteristics and complexity of the culture equipment.

[0055] Furthermore, the image of the culture vessel to be detected is input into the ROI localization engine to locate the corresponding ROI detection algorithm, and the ROI detection algorithm is used to detect at least one ROI region in the image of the culture vessel to be detected.

[0056] As mentioned above, Figure 7As shown, the ROI localization engine contains various ROI detection algorithms, including ordinary square dish ROI localization algorithm, filter membrane ROI localization algorithm, and test piece localization algorithm. It maps and matches based on the classification results of the major categories and obtains the matching ROI detection algorithm based on the ROI detection method and the input culture vessel image. It executes the ROI detection algorithm to obtain the ROI region coordinates and extracts high-resolution ROIs from the input culture vessel image based on the ROI region coordinates.

[0057] This solution's ROI localization engine has multiple pre-set ROI detection algorithms, as shown below:

[0058] Where R stands for ROI positioning engine. For the k-th ROI detection algorithm, This indicates the number of ROI algorithms.

[0059] It should be noted that the number of ROI detection algorithms may or may not be equal to the number of major categories. Typically, the number of ROI detection algorithms is equal to the number of major categories, because each major category corresponds to one ROI detection algorithm.

[0060] For the input image of the culture vessel to be detected, the mapping function Φ is used to match the corresponding ROI detection algorithm from the ROI localization engine, as shown below:

[0061] Then, the matched ROI detection algorithm is used to process the ROI coordinates of the culture vessel image to be detected, resulting in a list B of ROI region coordinates, defined as follows: ; in This indicates the number of ROI region coordinates in the list, each... It is a bounding box (x, y, width, height).

[0062] It should be noted that the ROI detection algorithm may output one or multiple ROI regions for the culture vessel image to be detected. For example, the ROI detection algorithm for filter plate returns two ROI regions (the ROI of the plate and the ROI of the filter).

[0063] Then, based on the ROI region, the corresponding ROI image is extracted from the image of the culture vessel to be detected to amplify the image features. Therefore, the ROI localization engine mentioned in this solution can be represented as:

[0064] Where I represents the image of the culture device to be detected, and I[b] represents the ROI image. In addition, regarding the culture vessel images to be detected that require fine classification, this scheme, after obtaining the ROI image of the corresponding culture vessel image to be detected, inputs the ROI image into the fine classification model to output the subclass classification result.

[0065] Regarding the "fine-classification model" in this scheme: The "fine-classification model" in this scheme is trained to output the subclass classification results and classification confidence of the culture vessel images. Specifically, images of various culture vessels containing the subclass classification results of labeled culture vessels are used as the training dataset, and the fine-classification model is trained using the training dataset.

[0066] Furthermore, this approach collects image sets of various culture vessels labeled with their subclass classification results as the training dataset. It should be noted that this approach collects ROI image sets of culture vessels requiring further subclassification and labels the corresponding subclass classification results as the training dataset.

[0067] In some embodiments, in addition to using the cropped images corresponding to the ROI detection algorithm, the ROI image set collected by this scheme also needs to be offset to different degrees from the standard ROI coordinates in order to collect richer ROI cropped images and reduce the requirements of the dedicated model on the accuracy of ROI coordinates.

[0068] In some embodiments, the collected training dataset is image-normalized before being input into the coarse classification model for training.

[0069] Furthermore, image standardization processing methods include, but are not limited to, size normalization, color space conversion (RGB to grayscale or retaining RGB), HSV enhancement, translation, flipping, perspective, and noise filtering.

[0070] Furthermore, the pre-trained fine classification model of this scheme includes a feature extraction network and a fine classifier, wherein the feature extraction network is trained to extract deep abstract features in the ROI image, and the fine classifier is trained to identify the subclass classification results of the ROI image.

[0071] Specifically, the ROI image is input into the feature extraction network of the fine classification model to extract deep abstract features. These deep abstract features are then input into the fine classifier for classification and recognition to obtain the subclass classification result of the ROI image. It should be noted that since the major category of the culture device image to be detected is known in step S1, the fine classification model corresponding to the major category can be selected in step S2.

[0072] It should be noted that, since the subclassification of some culture equipment requires the extraction of manual features, this solution additionally extracts manual features through a feature extractor for culture equipment that requires manual feature extraction, and incorporates the manual features into the fine classification model to predict the subclassification results.

[0073] Correspondingly, in step S2, if the major classification result of the ROI image shows a type that requires the extraction of handcrafted features, then the ROI image is input into the feature extraction network of the pre-trained fusion fine classification model to extract deep abstract features, the ROI image is input into the feature extractor to extract handcrafted features, the handcrafted features and deep abstract features are fused to obtain fused features, and the fused features are input into the fine classifier to output the subclass classification result. Otherwise, the ROI image is input into the feature extraction network of the pre-trained fusion fine classification model to extract deep abstract features, the deep abstract features are input into the fine classifier to output the subclass classification result.

[0074] In some embodiments, a type of handcrafted feature to be extracted is preset, and the ROI image is determined to be of the type that needs to be extracted by comparing the major classification results of the ROI image with the preset type of handcrafted feature to be extracted.

[0075] The feature extraction network in the fine-classification model of this scheme adopts a lightweight CNN convolutional neural network model to automatically learn and extract deep abstract features of the input image. As mentioned earlier, this scheme uses a large training dataset to train the above CNN convolutional neural network model to obtain a feature extraction network that meets the feature extraction requirements of this scheme. Similarly, this scheme uses training data of specific major categories to train the classifier to obtain the fine-classifier.

[0076] In some embodiments, if the classification result of the ROI image shows a type that requires the extraction of handcrafted features, the feature layer of the feature extraction network is used to concatenate or weightedly fuse deep abstract features and handcrafted features to obtain more comprehensive fused features.

[0077] Specifically, the ROI image is input into the feature extractor to obtain handcrafted features. The ROI image is then input into the feature extraction network of the pre-trained fusion fine classification model to output deep abstract features. The deep abstract features and handcrafted features are then fused to obtain fused features. The fused features are then input into the fine classifier to output the subclass classification results.

[0078] It should be noted that this scheme adopts a phased training model approach. That is, it uses images of various culture equipment, including the subclass classification results of labeled culture equipment, as the training dataset. After training the feature extraction network with the training dataset, the feature extraction network is fixed to complete the training of the feature extraction network. If the major classification result of the ROI image shows that the type of manual features need to be extracted, the manual features extracted by the feature extractor and the deep abstract features extracted by the feature extraction network are fused together, and then the classifier is trained to obtain the corresponding fine classifier. Otherwise, the deep abstract features extracted by the feature extraction network are directly used to train the corresponding fine classifier.

[0079] Specifically, the deep abstract features of the feature extraction network are concatenated or weighted and fused with handcrafted features at the feature layer to form a more comprehensive fused feature. Finally, through a fully connected layer and a softmax classifier, the probability distribution of the subclass is output, and the classification network is trained to obtain a fine classifier.

[0080] In some embodiments, handcrafted features include contour moments, Hu moments, color histograms, etc.

[0081] Furthermore, obtain the ROI image ROI(I, s) and the major classification result t. Based on the characteristics of the major classification result t, determine whether manual feature extraction is necessary. At this point, an indicator function can be defined. ,in Representation type Manual features need to be extracted. This indicates that it is not needed.

[0082] If handcrafted features need to be extracted, then the corresponding handcrafted features can be extracted using a feature extractor. The data is then input into the fusion fine-classification model, where the feature extraction network extracts deep abstract features. By combining handcrafted features and deep abstract features, a fused feature is obtained:

[0083] If manual feature extraction is not required, the deep abstract features are directly input into a regular fine classification model. In this case, the fused features are equal to the deep abstract features. ; The fused features are then input into a fine classifier to obtain the corresponding subclass classification results.

[0084] Specifically, such as Figure 8 As shown, the ROI image of the culture device and the fine classification type are input. The deep CNN model of the fine classification model is used to extract deep abstract features and determine whether it is necessary to extract handcrafted features. If so, the key handcrafted features are extracted using traditional algorithms. The handcrafted features and deep abstract features are fused to obtain fused features. The fused features are input into the fine classifier to output subclasses and confidence scores.

[0085] It should be noted that this scheme trains corresponding fine classifiers for different categories of culture equipment that require fine classification, i.e., a set of fine classification types. It contains multiple subsets, each corresponding to a dedicated fine-grained classifier. Each dedicated fine-grained classifier is responsible for classifying the samples within its subset in a fine-grained manner, assigning them to different subclasses. The number of subclasses for each dedicated classifier may vary. The set of dedicated classifiers is defined as follows: ; in This represents the number of specialized classifiers and the set of sub-classification types. The quantities are consistent.

[0086] As mentioned above, the major categories requiring further subcategorization recorded in the detailed classification list of this scheme are ordinary round dishes, filter membrane petri dishes, and test plates. Therefore, the subcategorization result of the culture vessel image to be tested corresponding to ordinary round dishes in step S2 is "ordinary round petri dish" or "contact plate petri dish"; the subcategorization result of the culture vessel image to be tested corresponding to filter membranes in step S2 is "mesh filter membrane" or "non-mesh filter membrane"; and the subcategorization result of the culture vessel image to be tested corresponding to test plates in step S2 is different brands or models.

[0087] It should be emphasized again that since step S2 of this scheme is for processing and analyzing ROI images, that is, the fine classification model analyzes the magnified key regions with pure features, step S2 can be made very small and precise, easier to train, extremely fast inference speed, and extremely high accuracy.

[0088] Regarding step S3 of this scheme, the processing steps of "inputting the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region, and extracting the ROI image from the image of the culture vessel to be detected according to the ROI region" are the same as those of step S2. The coarse classification model extracted in "inputting the ROI image into the pre-trained coarse classification model again to output the verification classification result" is also the same as the coarse classification model in step S1, so it will not be explained again.

[0089] It should be noted that the verification classification result output in step S3 is the same as the major category classification result.

[0090] As described above, the automatic classification method for microbial culture equipment images provided in this solution processes images of different types of culture equipment differently. For direct output types, the major classification results obtained in step S1 are directly output. For types requiring finer classification, the subcategories need to be further subdivided, and details are cropped and magnified using ROI positioning before being input into the finer classification model, with the subcategories being directly output as the final classification result. For types to be validated, due to insufficient confidence, details are cropped and magnified using ROI positioning before being input into the coarse classification model for further validation, with the classification result being directly output as the final classification result. The corresponding hierarchical relationships are expressed as follows:

[0091] in This indicates the final classification result.

[0092] Image classification process for different culture equipment, such as Figure 4 and Figure 6 As shown, Figure 4 This is a classification process for images of culture vessels, including ordinary round dishes, anomalous samples, and ordinary square dishes. The input images of culture vessels are identified as ordinary round dishes, anomalous samples, and ordinary square dishes by a coarse classification model. After further ROI localization of ordinary round dishes and anomalous samples, the ROI images of ordinary round dishes are input into a fine classification model to obtain the output results, while the ROI images of anomalous samples are input into a coarse classification model to obtain the output results.

[0093] Figure 5 It is a classification process for culture vessel images of filter membrane petri dishes. The input culture vessel image is identified as a filter membrane petri dish by the coarse classification model. After further ROI localization of the filter membrane petri dish, the ROI image of the filter membrane petri dish is input into the fine classification model to obtain the output result. Figure 6 It is a classification process for culture vessel images of test pieces. The input culture vessel image is identified as a test piece by the coarse classification model. After further ROI localization of the test piece, the ROI image of the test piece is input into the fine classification model to obtain the output result.

[0094] Example 2 This embodiment also provides an electronic device, see reference. Figure 9 It includes a memory 404 and a processor 402, the memory 404 storing a computer program and the processor 402 being configured to run the computer program to execute the above-described automatic classification method for images of microbial culture equipment.

[0095] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0096] The memory 404 may include a mass storage device 404 for data or instructions. For example, and not as a limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, a magnetic tape drive, or a Universal Serial Bus (USB) drive, or a combination of two or more of these.

[0097] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the automatic classification methods for images of microbial culture equipment in the above embodiments.

[0098] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402. The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device.

[0099] The input / output device 408 is used to input or output information. In this embodiment, the input information may be an image of the culture device to be detected, and the output information may be a major classification result, a minor classification result, or a verification classification result, etc.

[0100] Optionally, in this embodiment, the processor 402 can be configured to perform the following steps via a computer program: S1: Input the image of the culture vessel to be detected into the pre-trained coarse classification model to obtain the major classification result and classification confidence. If the major classification result shows that the current culture vessel needs to be further classified, then proceed to step S2; if the classification confidence is less than the set threshold, then proceed to step S3. S2: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region, extract the ROI image from the image of the culture vessel to be detected based on the ROI region, and input the ROI image into the pre-trained fine classification model to output the subclass classification result. S3: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region. Extract the ROI image from the image of the culture vessel to be detected based on the ROI region. Input the ROI image back into the pre-trained coarse classification model to output the verification classification result.

[0101] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0102] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0103] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. One or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted that any block in the logical flow of the figures may represent a program step, or interconnected logical circuitry, blocks and functions, or a combination of program steps and logical circuitry, blocks and functions. The software may be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0104] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An automatic image classification method for microbial culture equipment, characterized in that, Includes the following steps: S1: Input the image of the culture vessel to be detected into the pre-trained coarse classification model to obtain the major classification result and classification confidence. If the major classification result shows that the current culture vessel needs to be further classified, then proceed to step S2; if the classification confidence is less than the set threshold, then proceed to step S3. S2: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region, extract the ROI image from the image of the culture vessel to be detected based on the ROI region, and input the ROI image into the pre-trained fine classification model to output the subclass classification result. S3: Input the image of the culture vessel to be detected into the ROI localization engine to obtain the ROI region. Extract the ROI image from the image of the culture vessel to be detected based on the ROI region. Input the ROI image back into the pre-trained coarse classification model to output the verification classification result.

2. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, Images of various culture equipment, including the major classification results of labeled culture equipment, are used as the training dataset. A coarse classification model is trained using the training dataset.

3. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, A pre-defined subcategories list is provided, which records the major categories that need to be subcategorized. If the major category classification result of the culture vessel image to be detected belongs to the major category recorded in the subcategories list, then the culture vessel image to be detected belongs to the subcategories type and step S2 needs to be executed.

4. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, Based on the major classification results, it is determined whether the current culture equipment needs to be further classified. If it does not need to be further classified, the classification confidence score is then used to determine whether it is less than a set threshold.

5. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, The ROI localization engine has preset ROI detection algorithms for different categories of culture equipment. Based on the category classification result of the culture equipment image to be detected, the ROI localization engine matches the corresponding ROI detection algorithm for the major category.

6. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, The main classification results and verification classification results include any one of the following: ordinary round dishes, ordinary square dishes, filter membrane dishes, test pieces, and abnormal samples. The subclassification result for the culture vessel image to be tested corresponding to ordinary round dishes is "ordinary round dish" or "contact dish dish"; the subclassification result for the culture vessel image to be tested corresponding to filter membrane dishes is "mesh filter membrane" or "meshless filter membrane"; and the subclassification result for the culture vessel image to be tested corresponding to test pieces is different brands or models.

7. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, If the ROI image's major classification result indicates a type requiring handcrafted features, then the ROI image is input into the feature extraction network of a pre-trained fusion fine-classification model to extract deep abstract features. The ROI image is then input into a feature extractor to extract handcrafted features. The handcrafted features and deep abstract features are fused to obtain fused features, which are then input into a fine-classifier to output subclass classification results. Otherwise, the ROI image is input into the feature extraction network of a pre-trained fusion fine-classification model to extract deep abstract features, and the deep abstract features are then input into a fine-classifier to output subclass classification results.

8. The automatic image classification method for microbial culture equipment according to claim 7, characterized in that, Images of various culture equipment, including subclass classification results of labeled culture equipment, are used as the training dataset. The feature extraction network is trained using the training dataset and then fixed to complete the training. If the major classification result of the ROI image shows a type that requires manual feature extraction, the manual features extracted by the feature extractor and the deep abstract features extracted by the feature extraction network are fused, and the classifier is then trained to obtain the corresponding fine classifier. Otherwise, the deep abstract features extracted by the feature extraction network are directly used to train the corresponding fine classifier.

9. The automatic image classification method for microbial culture equipment according to claim 1, characterized in that, The coarse classification model includes a feature extraction module, a fully connected layer, and a classifier. Its feature extraction module includes multiple convolutional and pooling layers for automatically learning and extracting deep abstract features of the input image. The classifier uses softmax to output the major classification results of the image or to verify the classification results and classification confidence. The pre-trained fine-classification model includes a feature extraction network and a fine-classifier.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the automatic classification method for images of microbial culture equipment according to any one of claims 1 to 9.

Images