Microbial colony detection method, device, equipment, medium and product

Abstract

The invention discloses a microbial colony detection method, device and equipment, a medium and a product, and relates to the technical field of biological information.The method comprises the steps that to-be-detected images shot by a camera are obtained at intervals of preset time; preprocessing the to-be-detected image to obtain a target detection image; performing feature extraction on the target detection image to obtain feature maps of different levels; fusing the feature maps of different levels by adopting a top-down and transverse connection mode to obtain a multi-scale feature map; and identifying the multi-scale feature map to obtain the variety and the number of the microbial colonies in the culture dish. The detection efficiency of the microbial colony and the accuracy of the detection result are improved.

Description

Technical Field

[0001] The present application relates to the field of bioinformatics, and in particular to a method, device, equipment, medium and product for detecting microbial colonies. Background Art

[0002] In biological experiments, after culturing fungal microorganisms, the strains in the culture dish need to be read. The existing method is that the staff reads the culture dish through a microscope. During the reading process, the size and position of the colony are first read by the ruler on the microscope, and then the miscellaneous bacteria are identified and removed based on experience. Finally, the bacteria are picked up into the well plate with the help of a bacteria picking tool for subsequent experimental operations.

[0003] Given that the entire process described above has a long cycle time and is prone to human errors, there is an urgent need for a method that can improve the efficiency and accuracy of microbial colony detection. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the related art, the purpose of this application is to provide a microbial colony detection method, device, equipment, medium and product, which can improve the detection efficiency of microbial colonies and the accuracy of detection results.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In the first aspect, the present application provides a microbial colony detection method, which includes: acquiring an image to be detected taken by a camera at a preset time interval; preprocessing the image to be detected to obtain a target detection image; extracting features from the target detection image to obtain feature maps at different levels; fusing the feature maps at different levels in a top-down and lateral connection manner to obtain a multi-scale feature map; identifying the multi-scale feature map to obtain the type and quantity of microbial colonies in the culture dish.

[0007] Optionally, the extracting features of the target detection image to obtain feature maps of different levels includes: performing multi-layer convolution and pooling processing on the target detection image based on a pre-trained underlying convolutional network to extract different colony features of the target detection image to obtain feature maps of different levels.

[0008] Optionally, the top-down and lateral connection method is used to fuse the feature maps of different levels to obtain a multi-scale feature map, including: based on a pre-trained feature pyramid network model, upsampling layer by layer from the high-level feature maps of the feature maps of different levels, and fusing the upsampled feature maps with the corresponding bottom-level feature maps of the feature maps of different levels; while upsampling layer by layer and fusing the upsampled feature maps with the corresponding bottom-level feature maps of the feature maps of different levels, merging the upsampled high-level feature maps with the bottom-level feature maps of the same resolution in the feature maps of different levels to obtain the multi-scale feature map.

[0009] Optionally, the multi-scale feature map is identified to obtain the type and quantity of microbial colonies in the culture dish, including: generating colony candidate regions of different sizes from the multi-scale feature map based on a region proposal network, and predicting the probability of existence of a target in each colony candidate region and the position offset of the bounding box; mapping the colony candidate regions of different sizes to a feature map of a fixed size, and performing feature extraction on each of the mapped colony candidate regions to obtain a target feature map; classifying each of the colony candidate regions in the target feature map to obtain a classification result; adjusting the position and size of the bounding box in the target feature map based on a regressor to obtain a target bounding box; and obtaining the type and quantity of microbial colonies based on the classification result and the target bounding box.

[0010] Optionally, the training method of the pre-trained feature pyramid network includes: labeling the colony categories in the collected historical detection images to obtain a sample data set; performing image preprocessing on the sample data set to obtain a training data set; constructing a feature pyramid network model based on a pre-trained convolutional neural network; inputting the training data set into the feature pyramid network model for training to obtain a trained feature pyramid network model; using a K-fold cross-validation method to verify the performance of the trained feature pyramid network model, and when the average performance of the trained feature pyramid network model reaches a preset standard, the trained feature pyramid network model is obtained.

[0011] Optionally, the preprocessing of the image to be detected to obtain the target detection image includes: converting the image to be detected into a grayscale image, and denoising and standardizing the image to be detected converted into the grayscale image to obtain a standard detection image; binarizing the standard detection image to obtain the target detection image.

[0012] In a second aspect, the present application provides a microbial colony detection device, the microbial colony detection device comprising:

[0013] An acquisition module is used to acquire the image to be detected taken by the camera at a preset interval;

[0014] A preprocessing module, used for preprocessing the image to be detected to obtain a target detection image;

[0015] A feature extraction module is used to extract features from the target detection image to obtain feature maps at different levels;

[0016] A feature fusion module is used to fuse the feature maps of different levels in a top-down and lateral connection manner to obtain a multi-scale feature map;

[0017] The recognition module is used to recognize the multi-scale feature map to obtain the type and quantity of microbial colonies in the culture dish.

[0018] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the microbial colony detection methods described above.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the microbial colony detection methods described above.

[0020] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the microbial colony detection methods described above.

[0021] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0022] The present application provides a method, device, equipment, medium and product for detecting microbial colonies, which obtain a target detection image by preprocessing an image to be detected taken by a camera; obtain feature maps of different levels by extracting features from the target detection image; obtain a multi-scale feature map by fusing the feature maps of different levels in a top-down and horizontal connection manner; obtain the types and quantities of microbial colonies in a culture dish by identifying the multi-scale feature map; on the one hand, the types and quantities of microbial colonies in the culture dish are obtained by identifying the target detection image, which reduces dependence on operators, greatly reduces manual errors caused by operators, and improves the efficiency of microbial colony detection and the accuracy of detection results; on the other hand, the features of the target detection image are extracted to the greatest extent by a top-down and horizontal connection manner, and the information carried by the target detection image is expressed more comprehensively, thereby improving the accuracy of identifying the types and quantities of microbial colonies in the culture dish. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of a process for detecting a microbial colony provided in one embodiment of the present application;

[0025] Figure 2 A schematic diagram of the functional modules of a microbial colony detection device provided in one embodiment of the present application;

[0026] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] In an exemplary embodiment, Figure 1As shown, a microbial colony detection method is provided, which is executed by a computer device, specifically, it can be executed by a computer device such as a terminal or a server alone, or it can be executed by a terminal and a server together. In the embodiment of the present application, the method can be executed by a terminal with a computing function alone, wherein the microbial colony detection method can include the following steps S110 to S150. Specifically:

[0030] Step S110, acquiring the image to be detected taken by the camera at preset time intervals.

[0031] In an exemplary embodiment, the image to be detected is a culture dish in a temperature-controlled culture process captured by a camera, and the culture dish contains at least one type of microbial colonies.

[0032] It should be noted that the microbial colonies involved in the embodiments of the present application may be bacteria such as Escherichia coli, Staphylococcus aureus, etc. The preset time may be determined according to the shooting time or shooting cycle of the camera, and is not specifically limited in the present embodiment.

[0033] Step S120, preprocessing the image to be detected to obtain a target detection image.

[0034] In an example embodiment, the preprocessing of the image to be detected may include grayscale processing, binarization processing, and segmentation processing. By preprocessing the image to be detected, the target detection image can meet the processing requirements of the subsequent pre-trained underlying convolutional network, which facilitates the pre-trained underlying convolutional network to extract features in the image to be detected, thereby ensuring the accuracy of the output results of the subsequent pre-trained feature pyramid network model.

[0035] In a specific embodiment, the above step S120 may include steps S1201 to S1203, specifically:

[0036] Step S1201, converting the image to be detected into a grayscale image, and performing denoising and standardization processing on the image to be detected converted into the grayscale image to obtain a standard detection image;

[0037] Step S1202, binarizing the standard detection image to obtain a second detection image in black and white;

[0038] Step S1203, performing image segmentation on the second detection image to obtain a target detection image.

[0039] It can be understood that the RGB color image to be detected is converted into a grayscale image, and then a filter (such as median filtering, Gaussian filtering) is used to remove the noise in the grayscale image. Then, in order to enhance the contrast of the grayscale image and make the difference between the colony and the background more obvious, the grayscale image is contrast enhanced to obtain a standard detection image.

[0040] In order to distinguish the microbial colonies from the background of the culture dish, the standard detection image is converted into only two colors (usually black and white) to obtain the target detection image. Among them, the commonly used methods of binarization processing include global threshold method and local threshold method. The global threshold method selects a fixed threshold to divide the image into foreground and background, such as the Otsu method to automatically calculate the optimal threshold; the local threshold method uses different thresholds for different areas of the standard detection image, which is suitable for the case of uneven background brightness.

[0041] It should be noted that by performing grayscale processing, denoising, standardization processing and binarization processing on the image to be detected, the difference between the background composed of other substances in the culture dish and the microbial colonies can be increased, providing data support for subsequent feature extraction and recognition of the target detection image.

[0042] Step S130, extracting features from the target detection image to obtain feature maps at different levels.

[0043] In the example embodiment, feature extraction refers to the process of extracting effective information from the target detection image that helps to identify and classify microbial colonies. In this process, the target detection image will gradually generate feature maps of different levels after a series of convolution and pooling processes. The size of these feature maps of different levels decreases with the increase of network depth, and the semantic level of the features increases, that is, from specific texture and edge information to more abstract object descriptions.

[0044] In a specific embodiment, the above step S130 may include: performing multi-layer convolution and pooling processing on the target detection image based on a pre-trained underlying convolutional network, extracting different colony features of the target detection image, and obtaining feature maps of different levels.

[0045] Understandably, using a pre-trained convolutional neural network (such as ResNet) as the base network, feature maps are gradually extracted from the target detection image through multi-layer convolution and pooling operations. In the ResNet network, as the network layers increase, the spatial resolution of the feature map gradually decreases, but the semantic information of the features gradually increases. Low-level feature maps retain more spatial details (such as edges and textures), but less semantic information; high-level feature maps have richer semantic information (such as overall shape and high-level features of objects), but lower spatial resolution.

[0046] In step S140, feature maps at different levels are fused in a top-down and horizontal connection manner to obtain a multi-scale feature map.

[0047] In an exemplary embodiment, the top-down approach is to start from the highest level feature map, upsample layer by layer (usually using deconvolution or nearest neighbor interpolation) and fuse the feature maps of the lower layers. For example, given the feature map P of layer i i , the upsampled feature map P i+1 It is fused with the feature map of the i-th layer through element-wise addition or concatenation operation.

[0048] The lateral connection is that the feature map of each layer is horizontally connected with the corresponding low-level feature map in the backbone network in the top-down path. The lateral connection combines the semantically rich low-resolution feature map from the upper layer with the high-resolution feature map with rich spatial details to generate a multi-scale feature map.

[0049] In a specific embodiment, the above step S140 may include:

[0050] Step S1401, based on the pre-trained feature pyramid network model, up-sampling starts from the high-level feature maps of different levels of feature maps layer by layer, and fusing the up-sampled feature maps with the corresponding low-level feature maps of different levels of feature maps;

[0051] Step S1402, while upsampling layer by layer and fusing the upsampled feature map with the corresponding underlying feature map of the feature map at different levels, the upsampled high-level feature map is merged with the underlying feature map of the same resolution in the feature map at different levels to obtain a multi-scale feature map.

[0052] Understandably, the embodiment of the present application adopts a trained feature pyramid network model to obtain a multi-scale feature map; the specific trained feature pyramid network model introduces a top-down path, starting from the high-level feature maps of feature maps at different levels, upsampling layer by layer (usually using deconvolution or nearest neighbor interpolation) and fusing the feature maps of the lower layers; the feature maps at each layer are horizontally connected with the corresponding low-level feature maps in the backbone network of the trained feature pyramid network model in the top-down path; the horizontal connection combines the semantically rich low-resolution feature maps from the upper layer with the high-resolution feature maps with rich spatial details to generate a multi-scale feature map. The multi-scale feature map can process targets of different sizes at the same time, improving the detection effect of the trained feature pyramid network model.

[0053] Step S150, identifying the multi-scale feature map to obtain the type and quantity of the microbial colonies in the culture dish.

[0054] In a specific embodiment, the above step S150 may include steps S1501 to S1505, specifically:

[0055] Step S1501, based on the region proposal network (RPN), colony candidate regions of different sizes are generated from the multi-scale feature map, and the probability of the existence of an object in each colony candidate region and the position offset of the bounding box are predicted.

[0056] It should be noted that in some embodiments, the feature pyramid network model constructed in the embodiments of the present application includes a region proposal network RPN. The region proposal network RPN is a fully convolutional network that can generate a large number of candidate regions on feature maps of different scales in a sliding window manner, and predict the probability of the existence of a target and the position offset of the bounding box for each region.

[0057] Step S1502, mapping colony candidate regions of different sizes onto a feature map of a fixed size, and performing feature extraction on each mapped colony candidate region to obtain a target feature map.

[0058] It should be noted that in the embodiment of the present application, RoIPooling or RoIAlign technology can be used to map colony candidate regions of different sizes to a feature map of a fixed size. Mapping colony candidate regions of different sizes to a feature map of a fixed size can more accurately align the feature map and avoid quantization errors.

[0059] Step S1503, classify each colony candidate region in the target feature map to obtain a classification result.

[0060] Step S1504: Adjust the position and size of the bounding box in the target feature map based on the regressor to obtain the target bounding box.

[0061] Step S1505 , based on the classification result and the target bounding box, the type and quantity of the microbial colony are obtained.

[0062] In combination with the above embodiments, it can be understood that on the basis of the generated multi-scale feature map, colony candidate regions of different sizes are generated by a region proposal network (RPN). RPN predicts the probability of a target existence and the position offset of the bounding box for each candidate region. Afterwards, for each colony candidate region of different sizes, the bounding box of the colony is further accurately located through convolution and regression operations, so that the predicted bounding box is closer to the actual colony position. Afterwards, for each colony candidate region, a classifier such as Faster R-CNN is used for classification, and a regressor is used to adjust the position and size of the bounding box so that it more accurately surrounds the colony. The classifier assigns a category label (such as Escherichia coli, Staphylococcus aureus, etc.) to each colony candidate region, and the regressor is used to correct the position of the bounding box. Afterwards, when the trained feature pyramid network model recognizes a colony candidate region when the probability exceeds a certain threshold, it is determined that there is a colony in the colony candidate region, and the colony will be counted in the total number. After that, non-maximum suppression (NMS) can be used to eliminate overlapping bounding boxes and retain the most likely colony candidate regions. Finally, the trained feature pyramid network model outputs the number of identified colonies and the category of each colony.

[0063] In addition, in some embodiments, the location information (given by a bounding box), category, and confidence score of each colony may also be output.

[0064] For example, the target detection image is input into the trained feature pyramid network model to generate a series of feature maps of different scales; on the feature map of each scale, a series of colony candidate regions are generated through the region proposal network; for each colony candidate region, the trained feature pyramid network model outputs the probability that it belongs to the colony and the adjustment parameters of the bounding box; a recognition threshold (for example, 0.85) is set, and when the trained feature pyramid network model predicts that the probability of a colony in the current colony candidate region is greater than or equal to 0.85, it is determined that there is a colony in the colony candidate region; the non-maximum suppression algorithm is applied to remove overlapping bounding boxes; and the final confirmed colony position is output, including the bounding box coordinates, category label, and confidence score.

[0065] It should be noted that, in the embodiment of the present application, a softmax function is used for multi-class classification, and linear regression is used to adjust the position and size of the bounding box. The RPN network is used to generate candidate colony regions and provide the capabilities of regional classification and bounding box regression. In addition, the preset threshold is dynamically set according to the specific needs of clinical applications, aiming to balance the recognition accuracy and false positive rate, and the embodiment of the present application does not make specific restrictions.

[0066] The embodiment of the present application provides a method for detecting microbial colonies. The method comprises the following steps: obtaining a target detection image by preprocessing an image to be detected taken by a camera; obtaining feature maps of different levels by extracting features from the target detection image; obtaining a multi-scale feature map by fusing feature maps of different levels in a top-down and horizontal connection manner; and obtaining the types and quantities of microbial colonies in a culture dish by identifying the multi-scale feature map. On the one hand, the types and quantities of microbial colonies in the culture dish are obtained by identifying the target detection image, thereby reducing dependence on operators, greatly reducing manual errors caused by operators, and improving the efficiency of microbial colony detection and the accuracy of detection results. On the other hand, the features of the target detection image are extracted to the greatest extent by a top-down and horizontal connection manner, and the information carried by the target detection image is expressed more comprehensively, thereby improving the accuracy of identifying the types and quantities of microbial colonies in the culture dish.

[0067] In another exemplary embodiment of the present application, in order to ensure the accuracy of microbial colony identification, it is necessary to train the constructed feature pyramid network model to obtain a pre-trained feature pyramid network model, and the training method may also include the following steps S210 to S240. Among them:

[0068] Step S210 , labeling the colony categories in the collected historical detection images to obtain a sample data set.

[0069] Step S220, performing image preprocessing on the sample data set to obtain a training data set;

[0070] Step S230, constructing a feature pyramid network model based on the pre-trained convolutional neural network;

[0071] Step S240, training the pre-trained feature pyramid network model with the validation set in the sample data set to obtain a trained feature pyramid network model;

[0072] Step S250, using a K-fold cross-validation method to verify the performance of the trained feature pyramid network model, when the average performance of the trained feature pyramid network model reaches a preset standard, a trained feature pyramid network model is obtained.

[0073] As can be understood from the above embodiments, the historical detection image is an image obtained by using a high-definition camera to regularly shoot the culture dish during the temperature-controlled culture process. After the historical detection image is collected, professional and technical personnel mark the colony categories in the historical detection image; the marking method includes but is not limited to bounding box marking (drawing a rectangular box for each colony to determine the position and range) and classification marking (assigning a category label to each colony, such as Escherichia coli, Staphylococcus aureus, etc.). After that, the labeled historical detection image is subjected to grayscale processing, binarization processing, and segmentation processing to obtain a training data set.

[0074] Based on a pre-trained convolutional neural network (such as ResNet), a feature pyramid model is formed by adding top-down paths and lateral connections. Specifically, the created feature pyramid network model may include an input layer, an intermediate layer, and an output layer; the input layer includes a pre-trained convolutional neural network, the intermediate layer includes a feature fusion network, a region proposal network, and a RoIPooling network; the output layer includes a classifier and a regressor. After that, the model is trained using an annotated image dataset. During the training process, the feature pyramid model learns the characteristics of different colonies and minimizes the prediction error by adjusting the network weights. During training, the training dataset is fed into the feature pyramid model, and the feature pyramid model outputs the predicted colony location and category. The parameters of the feature pyramid model are updated by comparing the difference (loss function) between the prediction of the feature pyramid model and the actual annotation. At the end of each training cycle (epoch), the validation set in the training dataset is used to evaluate the current performance of the model in order to adjust the hyperparameters or stop the training early to prevent overfitting.

[0075] In addition, in order to improve the stability of feature pyramid model prediction, the K-fold cross validation method can be used. Divide the training data set into K subsets, use one of the subsets as the validation set each time, and the rest as the training set, cycle K times, and finally take the average value as the validation performance of the feature pyramid model. If the average performance index is low or the variance is large, it is necessary to adjust the structure, hyperparameters or data preprocessing of the feature pyramid model. If the average performance index reaches the expected level (such as the accuracy reaches a certain standard) and the variance is small, it means that the feature pyramid model is well trained and can be put into practical application.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a microbial colony detection device for implementing the microbial colony detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more microbial colony detection device embodiments provided below can refer to the limitations of the microbial colony detection method above, and will not be repeated here.

[0077] In an exemplary embodiment, Figure 2 As shown, a microbial colony detection device is provided. The microbial colony detection device 200 includes: an acquisition module 210, a preprocessing module 220, a feature extraction module 230, a feature fusion module 240 and an identification module 250. Specifically:

[0078] The acquisition module 210 is used to acquire the image to be detected taken by the camera at a preset time interval;

[0079] The preprocessing module 220 is used to preprocess the image to be detected to obtain a target detection image;

[0080] The feature extraction module 230 is used to extract features from the target detection image to obtain feature maps at different levels;

[0081] The feature fusion module 240 is used to fuse feature maps at different levels in a top-down and horizontal connection manner to obtain a multi-scale feature map;

[0082] The identification module 250 is used to identify the multi-scale feature map to obtain the type and quantity of the microbial colonies in the culture dish.

[0083] As an optional implementation, the feature extraction module 230 is also used to perform multi-layer convolution and pooling processing on the target detection image based on a pre-trained underlying convolutional network to extract different colony features of the target detection image and obtain feature maps at different levels.

[0084] As an optional implementation, the feature fusion module 240 is also used to upsample layer by layer starting from high-level feature maps of feature maps at different levels based on a pre-trained feature pyramid network model, and fuse the upsampled feature maps with the corresponding bottom-level feature maps of feature maps at different levels; while upsampling layer by layer and fusing the upsampled feature maps with the corresponding bottom-level feature maps of feature maps at different levels, the upsampled high-level feature maps are merged with the bottom-level feature maps of the same resolution in the feature maps at different levels to obtain a multi-scale feature map.

[0085] As an optional implementation, the above-mentioned microbial colony detection device is also used to label the colony categories in the collected historical detection images to obtain a sample data set; perform image preprocessing on the sample data set to obtain a training data set; construct a feature pyramid network model based on a pre-trained convolutional neural network; input the training data set into the feature pyramid network model for training to obtain a trained feature pyramid network model; use K-fold cross-validation to verify the performance of the trained feature pyramid network model, and when the average performance of the trained feature pyramid network model reaches a preset standard, a trained feature pyramid network model is obtained.

[0086] As an optional implementation, the preprocessing module 220 is also used to convert the image to be detected into a grayscale image, and to perform denoising and standardization on the image to be detected converted into the grayscale image to obtain a standard detection image; and to perform binarization on the standard detection image to obtain a target detection image.

[0087] Among them, by implementing this implementation method, on the one hand, the type and number of microbial colonies in the culture dish are obtained by identifying the target detection image, thereby reducing dependence on operators, greatly reducing human errors caused by operators, and improving the efficiency of microbial colony detection and the accuracy of detection results; on the other hand, through top-down and lateral connection methods, the characteristics of the target detection image are extracted to the greatest extent, and the information carried by the target detection image is expressed more comprehensively, thereby improving the accuracy of identifying the type and number of microbial colonies in the culture dish.

[0088] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store microbial colony detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a microbial colony detection method is implemented.

[0089] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0091] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0092] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0093] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0094] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0095] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0096] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting microbial colonies, characterized in that: The microbial colony detection method comprises: Acquire the image to be detected taken by the camera at a preset time interval; Preprocessing the image to be detected to obtain a target detection image; Extracting features from the target detection image to obtain feature maps at different levels; The feature maps at different levels are fused in a top-down and lateral connection manner to obtain a multi-scale feature map; The multi-scale feature map is identified to obtain the types and quantities of microbial colonies in the culture dish.

2. The microbial colony detection method according to claim 1, characterized in that: The feature extraction of the target detection image to obtain feature maps at different levels includes: Based on the pre-trained underlying convolutional network, multi-layer convolution and pooling processing is performed on the target detection image to extract different colony features of the target detection image to obtain the feature maps of different levels.

3. The microbial colony detection method according to claim 1, characterized in that: The method of fusing the feature maps at different levels in a top-down and horizontal connection manner to obtain a multi-scale feature map includes: Based on a pre-trained feature pyramid network model, upsampling is started layer by layer from the high-level feature maps of the feature maps at different levels, and the upsampled feature maps are fused with the corresponding low-level feature maps of the feature maps at different levels; While upsampling layer by layer and fusing the upsampled feature map with the corresponding underlying feature map of the feature map at different levels, the upsampled high-level feature map is merged with the underlying feature map of the same resolution in the feature map at different levels to obtain the multi-scale feature map.

4. The microbial colony detection method according to claim 1, characterized in that: The multi-scale feature map is identified to obtain the type and quantity of microbial colonies in the culture dish, including: Generate colony candidate regions of different sizes from the multi-scale feature map based on a region proposal network, and predict the probability of an object existing in each colony candidate region and the position offset of the bounding box; Mapping the colony candidate regions of different sizes onto a feature map of a fixed size, and performing feature extraction on each of the mapped colony candidate regions to obtain a target feature map; Classify each of the colony candidate regions in the target feature map to obtain a classification result; Adjusting the position and size of the bounding box in the target feature map based on the regressor to obtain a target bounding box; Based on the classification result and the target bounding box, the type and quantity of the microbial colony are obtained.

5. The microbial colony detection method according to claim 3, characterized in that: The training method of the pre-trained feature pyramid network includes: Label the colony categories in the collected historical detection images to obtain a sample data set; Performing image preprocessing on the sample data set to obtain a training data set; Construct a feature pyramid network model based on a pre-trained convolutional neural network; Inputting the training data set into the feature pyramid network model for training to obtain a trained feature pyramid network model; The performance of the trained feature pyramid network model is verified by using a K-fold cross validation method. When the average performance of the trained feature pyramid network model reaches a preset standard, the trained feature pyramid network model is obtained.

6. The method for detecting microbial colonies according to claim 1, characterized in that: The preprocessing of the image to be detected to obtain the target detection image includes: Converting the image to be detected into a grayscale image, and performing denoising and standardization processing on the image to be detected converted into the grayscale image to obtain a standard detection image; The standard detection image is binarized to obtain the target detection image.

7. A microbial colony detection device, characterized in that: The microbial colony detection device comprises: An acquisition module is used to acquire the image to be detected taken by the camera at a preset interval; A preprocessing module, used for preprocessing the image to be detected to obtain a target detection image; A feature extraction module is used to extract features from the target detection image to obtain feature maps at different levels; A feature fusion module is used to fuse the feature maps of different levels in a top-down and lateral connection manner to obtain a multi-scale feature map; The recognition module is used to recognize the multi-scale feature map to obtain the type and quantity of microbial colonies in the culture dish.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the microbial colony detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the microbial colony detection method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the microbial colony detection method according to any one of claims 1 to 6 are implemented.

Images