A colony counting method and device

Abstract

The present application discloses a colony counting method, which obtains an image to be processed taken for the colony. The image to be processed is segmented at multiple scales to obtain first images corresponding to different segmentation scales. For the first image corresponding to each segmentation scale, a feature map corresponding to the first image is obtained, and the feature map is weighted according to the attention mechanism to obtain the first feature map. Then, the colony is detected based on the first feature maps corresponding to all segmentation scales to obtain the number of colonies. That is, the present application integrates multi-scale and attention mechanism when performing colony detection. The introduction of multi-scale can assist in capturing detailed information and global information in the image. The introduction of attention mechanism can dynamically adjust the degree of attention to information at different scales, so as to better capture the diversity and complexity in the image, thereby providing more feature information for the detection of the number of colonies and improving the accuracy of the detection of the number of colonies.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a colony counting method and device. Background Art

[0002] Colony counting is an important experimental procedure in fields such as microbiology, medicine and food science, and is often used to assess the concentration of microorganisms in a sample.

[0003] The current colony counting methods mainly include manual counting, image processing technology, etc. Among them, the manual counting method is time-consuming and labor-intensive and is easily affected by human errors; although image processing technology has improved efficiency, the current image segmentation and counting methods still have problems such as insufficient accuracy and long processing time. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a colony counting method and device to improve the accuracy of colony counting.

[0005] To solve the above problems, the technical solutions provided in the embodiments of the present application are as follows:

[0006] In a first aspect of the present application, a colony counting method is provided, the method comprising:

[0007] Acquire an image to be processed, where the image to be processed is an image taken of a culture dish containing bacterial colonies;

[0008] Performing multi-scale segmentation on the image to be processed to obtain a plurality of first images, wherein the multi-scale includes at least two different segmentation scales, and one first image corresponds to one segmentation scale;

[0009] For a first image corresponding to any segmentation scale, a feature map corresponding to the first image is obtained, and a weighted process is performed on the feature map based on an attention mechanism to obtain a first feature map;

[0010] The colonies are detected based on the first feature maps corresponding to all segmentation scales to obtain the number of colonies.

[0011] In a second aspect of the present application, a colony counting device is provided, the device comprising:

[0012] An acquisition unit, used for acquiring an image to be processed, wherein the image to be processed is an image taken of a culture dish containing bacterial colonies;

[0013] A processing unit, configured to perform multi-scale segmentation on the image to be processed to obtain a plurality of first images, wherein the multi-scale segmentation includes at least two different segmentation scales, and one first image corresponds to one segmentation scale;

[0014] The processing unit is further configured to obtain, for a first image corresponding to any segmentation scale, a feature map corresponding to the first image, and perform weighted processing on the feature map based on an attention mechanism to obtain a first feature map;

[0015] The detection unit is used to detect the colonies based on the first feature graphs corresponding to all segmentation scales to obtain the number of colonies.

[0016] In a third aspect of the embodiments of the present application, an electronic device is provided, including: a processor, a memory;

[0017] The memory is used to store computer-readable instructions or computer programs;

[0018] The processor is used to read the computer-readable instructions or the computer program so that the electronic device implements the colony counting method described in the first aspect.

[0019] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a device, the device executes the colony counting method described in the first aspect.

[0020] In a fifth aspect of the present application, a computer program product is provided. When the computer program product is run on a computer, the computer executes the colony counting method described in the first aspect.

[0021] It can be seen that the embodiments of the present application have the following beneficial effects:

[0022] In the present application, in order to realize colony counting, an image to be processed taken for the colony is obtained. The image to be processed is segmented at multiple scales to obtain first images corresponding to different segmentation scales. For the first image corresponding to each segmentation scale, a feature map corresponding to the first image is obtained, and the feature map is weighted according to the attention mechanism to obtain a first feature map. Then, the colony is detected based on the first feature map corresponding to all segmentation scales to obtain the number of colonies. That is, when performing colony detection, the present application integrates multi-scale and attention mechanism. The introduction of multi-scale can assist in capturing detailed information and global information in the image. The introduction of attention mechanism can dynamically adjust the degree of attention to information at different scales, so as to better capture the diversity and complexity in the image, thereby providing more feature information for the detection of the number of colonies and improving the accuracy of the detection of the number of colonies. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a colony counting method provided in an embodiment of the present application;

[0024] Figure 2A colony counting framework diagram provided in an embodiment of the present application;

[0025] Figure 3a A structural diagram of a colony segmentation model provided in an embodiment of the present application;

[0026] Figure 3b A structural diagram of a colony boundary segmentation model provided in an embodiment of the present application;

[0027] Figure 3c A structural diagram of a colony detection model provided in an embodiment of the present application;

[0028] Figure 4 A structural diagram of a colony counting device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Currently, colony counting methods based on image processing still have the problem of insufficient accuracy. For example, the threshold-based segmentation method is easily affected by uneven lighting, and although deep learning methods have made significant progress, there are still challenges in dealing with adherent colonies.

[0031] Based on this, the present application provides a colony counting method, which segments colonies through multi-scale feature fusion. For colonies with large and small bacteria or adhesion areas, the boundaries of the adhesion areas can be accurately segmented by combining ESPNet with adaptive convolution, and finally the improved yolov8 is used for colony detection, and the final counting results are obtained by combining Soft-NMS. This method can achieve faster and more accurate colony counting results.

[0032] When implementing the technical solution of the present application, three network models are involved, namely a colony segmentation model, a boundary segmentation model and a detection model.

[0033] The colony segmentation model is used to perform colony segmentation on the small image obtained through multi-scale segmentation to obtain a preliminary segmentation result of the colony area. The preliminary segmentation result may include segmented areas where large and small bacteria and / or adhesion parts exist. The colony segmentation model may be a U-Net network that integrates a multi-scale Attention module, and the U-Net network can segment colonies of different sizes and shapes. The U-Net network is an end-to-end image segmentation model based on a convolutional neural network, and its main feature is its symmetrical encoder-decoder structure. The encoder is responsible for extracting image features, while the decoder reconstructs and generates the segmented image.

[0034] The boundary segmentation model is used to finely segment the boundaries of segmented areas where large and small bacteria and / or adhesion parts exist. Among them, the boundary segmentation model can be an ESPNet network integrated with adaptive convolution. ESPNet is a lightweight segmentation model that uses efficient depth-separable convolution to reduce computational overhead and introduces a hierarchical feature decoder to restore details. Adaptive convolution dynamically adjusts the shape and weight of the convolution kernel to adapt to local features in the input image. This enables it to handle colonies of various shapes and sizes, including adhesion colonies and a variety of colony types. The addition of adaptive convolution can further improve the ability to capture details and boundaries, thereby enhancing the accuracy of feature extraction.

[0035] The detection model is used to identify the location and size of the colony in the input image. The detection model can be an improved yolov8 network model, which uses FasterBlock of FasterNet to improve the feature extraction network and uses Dysample as a dynamic upsampler to improve the detection performance.

[0036] To obtain the above three models, we first need to prepare a data set and train them using the data set. Specifically, the following process can be included:

[0037] 1. Dataset Preparation

[0038] (1) Data collection and preprocessing

[0039] Data collection: Acquire high-resolution (e.g., 4K) images of the culture dishes that cover different types and densities of colonies.

[0040] Data annotation: annotate the colonies in the image, including the location, boundary and category label of each colony. The category label means that the object included in the image is "bacteria".

[0041] Data augmentation: Use image augmentation techniques (such as rotation, scaling, flipping, brightness adjustment, etc.) to increase the diversity of the dataset and thus improve the generalization ability of the model.

[0042] (2) Segmentation and resizing

[0043] Segmentation method (such as the 4 segmentation scales provided below):

[0044] a. Direct resize: directly resize the original image to 512*512 size.

[0045] b. 2x2 cutting: Cut the original image into 4 small images, the size of each small image is 1 / 2 of the original image size, and then resize each small image to 512*512.

[0046] c. 4x4 cutting: Cut the original image into 16 small images, the size of each small image is 1 / 4 of the original image size, and then resize each small image to 512*512.

[0047] d. 8x8 cutting: Cut the original image into 64 small images, each of which is 1 / 8 of the original image size, and then resize each small image to 512*512. Image processing: Standardize the segmented image to ensure that the pixel value of each small image is within a uniform range, which is convenient for model training and prediction.

[0048] 2. Model Training and Optimization

[0049] (1) Colony segmentation model

[0050] Model selection: Select the U-Net network as the basic architecture, combined with the multi-scale Attention module to improve the segmentation ability of colonies of different scales.

[0051] Training process:

[0052] a. Input data: The cut 512*512 small image is used as input, and the annotation information is used as the target output.

[0053] b. Loss function: Use cross entropy loss function or Dice coefficient to optimize model performance.

[0054] c. Training strategy: Use strategies such as data enhancement, learning rate adjustment, and early stopping to optimize the training process to improve the convergence speed and generalization ability of the model.

[0055] (2) Colony boundary segmentation model

[0056] Model selection: Based on the ESPNet network, combined with an adaptive convolution module, the boundaries of adherent colonies are refined and segmented.

[0057] Training process:

[0058] a. Input data: The results of U-Net segmentation with large and small bacteria and adhesion parts are used as input, and training is performed in combination with the labeled boundary information.

[0059] b. Loss function: Use boundary loss function (such as boundary-aware loss) to improve the accuracy of boundary segmentation.

[0060] (3) Improved yolov8 detection model

[0061] Model selection: Improved yolov8 model, using fasterBlock of fasterNet to improve feature extraction network, and using Dysample as a dynamic upsampler to improve detection performance.

[0062] Training process:

[0063] a. Input data: Use the cut 512x512 small image and the corresponding colony annotation data for training.

[0064] b. Loss function: Use the target detection loss function (such as the YOLO loss function) to optimize the detection performance of the model.

[0065] The specific implementation of this application will be explained below in combination with the above three models and specific embodiments.

[0066] See also Figure 1 , which is a flow chart of a colony counting method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0067] S101: Acquire an image to be processed.

[0068] Among them, the image to be processed is an image taken of a culture dish containing colonies. The colonies in the culture dish may have significant differences in shape and size. The shape of the colonies can range from a regular circle to an irregular heterogeneity, and the size can range from tiny dots to larger clumps.

[0069] In some embodiments, in order to prevent images taken under different light sources from affecting the counting results, for the same culture dish, the images to be processed may include images taken under different light sources. For example, one image is taken when the light source above the culture dish is turned on, and another image is taken when the light source below the culture dish is turned on.

[0070] S102: Perform multi-scale segmentation on the image to be processed to obtain a plurality of first images.

[0071] Among them, the multi-scale includes at least two different segmentation scales, and the different segmentation scales can be preset. For example, directly resize the image to be processed to 512*512 to obtain the first image; cut the image to be processed into 4 small images according to 2*2, and then resize them to 512*512 respectively to obtain the first image; cut the image to be processed into 16 small images according to 4*4, and then resize them to 512*512 respectively to obtain the first image; cut the image to be processed into 64 small images according to 8*8, and then resize them to 512*512 respectively to obtain the first image. Among them, one first image corresponds to one segmentation scale, and one first image includes multiple small images.

[0072] It should be noted that when the image to be processed includes at least two images to be processed under different light sources, multi-scale segmentation is performed for each image to be processed to obtain multiple first images corresponding to the image to be processed. For example, the images to be processed are image 1 and image 2, and two scale segmentations are performed on the images to be processed, namely 2*2 segmentation and 4*4 segmentation. For image 1, when 2*2 segmentation is performed, the first image 1-1 is obtained; when 4*4 segmentation is performed, the first image 1-2 is obtained; for image 2, when 2*2 segmentation is performed, the first image 2-1 is obtained; when 4*4 segmentation is performed, the first image 2-2 is obtained.

[0073] S103: For a first image corresponding to any segmentation scale, obtain a feature map corresponding to the first image, and perform weighted processing on the feature map based on an attention mechanism to obtain a first feature map.

[0074] After obtaining the first image corresponding to each segmentation scale, feature extraction is performed on each first image to obtain a multi-scale feature map. For the feature map corresponding to each segmentation scale, the feature map is weighted according to the attention mechanism to obtain the first feature map.

[0075] Specifically, this embodiment uses a U-Net network model that integrates a multi-scale Attention module to adjust the degree of attention to features of different scales and enhance the learning ability of details and global features. Feature extraction: Extract feature maps of multiple scales from multiple first images. The feature maps are generated through different convolutional layers or downsampling layers to capture the details and global information in the image. For example, the input image is I, and the image I belongs to R H×W×C， , where H and W are the height and width of the image, respectively, and C is the number of channels. After feature extraction, feature maps of multiple scales {F1, F2, …, F n}, F i Represents the feature map of the i-th scale (if following the above image segmentation example, i is equal to 4).

[0076] Attention calculation: Attention operations are performed on feature maps of different scales. The Attention mechanism dynamically adjusts the degree of attention paid to different feature maps according to the importance of the features, thereby enhancing the model's responsiveness to key feature maps.

[0077] Among them, the Attention mechanism includes three different vectors, namely:

[0078] Query: The query vector represents the target to focus on or retrieve. In natural language processing tasks, the query can be a target language word that is used to match the key vector to find relevant information.

[0079] Key: The key vector represents the source to be matched or compared with the query vector. In translation tasks, the key can be the source language word that is compared with the query vector to find the best match.

[0080] Value: The value vector represents the information to be weighted and summed according to the matching degree between the query vector and the key vector. In translation tasks, the value can be the embedding vector of the source language word, and the value is weighted and summed according to the matching degree between the query and the key to generate the final output.

[0081] Use the convolutional layer to generate queries, keys, and values, as shown in formulas (1)-(3):

[0082]

[0083] Among them, W Q , W K , W V are the weight matrices for query, key, and value respectively;

[0084] The attention weight is calculated using the dot product as shown in formula (4):

[0085]

[0086] Among them, d k is the dimension of the key, and the softmax function is used to normalize the attention weights into a probability distribution.

[0087] Finally, the attention weights are applied to the values ​​to obtain the weighted features, as shown in formula (5):

[0088]

[0089] Among them, F' i Represents the first feature map.

[0090] It should be noted that when the image to be processed includes images taken under different light sources, multi-scale segmentation is performed for the image under any light source to obtain multiple first images. The first images corresponding to the same segmentation scale under different light sources are merged, and then feature extraction is performed on the merged image to obtain a feature map, which includes all features in the first images corresponding to the same segmentation scale under different light sources.

[0091] S104: Detect the colonies based on the first feature graphs corresponding to all segmentation scales to obtain the number of colonies.

[0092] After obtaining the first feature maps corresponding to different segmentation scales, the image to be processed is segmented according to the first feature maps corresponding to all segmentation scales, and the image obtained by the segmentation is input into the detection model to obtain the number of colonies.

[0093] Specifically, weighted fusion is performed based on the first feature maps corresponding to all segmentation scales to obtain a target feature map; the image to be processed is segmented based on the target feature map to obtain a segmented image; and the segmented image is detected to determine the number of colonies.

[0094] Specifically, the feature maps adjusted by Attention are fused to obtain the final multi-scale feature representations. These feature representations combine information at different scales and improve the ability to understand complex scenes.

[0095] The weighted feature maps of all scales {F1', F2', ..., F n '} can be fused by weighted averaging, as shown in formula (6):

[0096]

[0097] Among them, α i is the weight of each scale obtained through training.

[0098] In this embodiment, if the image to be processed does not include large and small bacteria and / or colony adhesion parts, the image to be processed is directly segmented using the target feature map to obtain a segmented image, and colony detection is performed on the segmented image to obtain the number of colonies.

[0099] Specifically, the detection model can be used to detect the colonies in the segmented image to obtain the number of colonies. The detection model can be an improved yolov8 network model, which includes an improved feature extraction network and a Dysample dynamic upsampler using FasterBlock in the FasterNet network. That is, the segmented image is detected using the improved yolov8 network model to obtain the number of colonies.

[0100] In the task of colony segmentation and counting, the morphology and boundaries of the colonies may be very complex. The introduction of FasterBlock can extract these features more finely and improve the accuracy of detection, especially the processing ability of small and overlapping colonies. At the same time, FasterBlock reduces the amount of calculation and parameters through an efficient convolution structure, and can provide higher computational efficiency when processing a large number of colony images. This is especially important for real-time applications or large-scale image data processing.

[0101] In FasterBlock, Depthwise SeparableConvolution, Channel Attention Mechanism and other modules are usually combined to improve the efficiency and accuracy of feature extraction.

[0102] Depthwise separable convolution requires calculating depthwise convolution and pointwise convolution separately, as shown in formulas (7) and (8):

[0103]

[0104] Among them, W i,j,c is the element of the depth convolution kernel, X h+i,w+j,c is the value of the input feature map, K h and K w are the height and width of the convolution kernel respectively;

[0105]

[0106] Among them, W c,c’ is the weight of the point-by-point convolution kernel, C in is the number of input channels.

[0107] Dysample is a dynamic upsampler that can better handle detail changes in images and improve the detail recovery ability of segmentation results by adaptively adjusting the size of feature maps during the upsampling process, especially in the case of colonies with rich details or a large number of adhesions, thereby improving the accuracy and robustness of colony detection.

[0108] Combined with FasterBlock, the overall calculation process is shown in formula (9):

[0109]

[0110] Among them, DepthwiseConv is a deep convolution operation, PointwiseConv is a point-by-point convolution operation, SE is a channel attention mechanism, and Dysample is a dynamic upsampling operation.

[0111] If the image to be processed includes large and small bacteria and / or the adhesion part of the colony, the first segmented image and the second segmented image are obtained when the image to be processed is segmented using the target feature map. The first segmented image does not include large and small bacteria and the adhesion part of the colony, and the second segmented image includes large and small bacteria and / or the adhesion of the colony. Large and small bacteria refer to one bacteria containing one or more bacteria, and colony adhesion refers to the adhesion part between different bacteria. Colony adhesion can be identified by the connected domain; large and small bacteria can be identified by the area of ​​the bacteria, and the area of ​​large and small bacteria is usually larger than that of normal bacteria.

[0112] After obtaining the first segmented image and the second segmented image, the first segmented image and the second segmented image are detected respectively to obtain the number of colonies. Specifically, the colonies in the first segmented image are directly detected to obtain the first colony information, which includes the first colony number and the first colony position. The first colony position refers to the position of each bacterium included in the first segmented image. Specifically, the improved yolov8 model can be used to detect the colonies in the first segmented image to obtain the first colony information.

[0113] For the second segmented image, first perform boundary segmentation on the colonies in the second segmented image to obtain a target image, perform multi-scale segmentation on the target image to obtain multiple second images; for the second image corresponding to any segmentation scale, detect the second image to obtain the second colony information. Among them, one second image corresponds to one segmentation scale, and the second image includes multiple small images. The second colony information includes the second colony number and the second colony position, the second colony position refers to the position of the bacteria included in the second image, and the second colony number refers to the total number of bacteria included in the second image. Specifically, the improved yolov8 model can be used to detect the colonies in the second segmented image to obtain the second colony information.

[0114] Among them, when performing boundary segmentation, ESPNet combined with adaptive convolution can be used to finely segment the boundaries of heavily adhered areas. ESPNet is a lightweight segmentation model that uses efficient depth-separable convolution to reduce computational overhead and introduces a hierarchical feature decoder to restore details. Adaptive convolution dynamically adjusts the shape and weight of the convolution kernel to adapt to local features in the input image. This enables it to handle colonies of various shapes and sizes, including adherent colonies and a variety of colony types. The addition of adaptive convolution can further improve the ability to capture details and boundaries, thereby enhancing the accuracy of feature extraction.

[0115] The core of adaptive convolution is to dynamically generate convolution kernels. is the input feature map (the feature map corresponding to the second segmented image), K is the predicted optimal convolution kernel, and F' is the output feature map after the convolution operation. The mathematical expression of adaptive convolution is as shown in formulas (10)-(12):

[0116]

[0117] Among them, K int is the convolution sum, Winit is the learned weight matrix, W1 and b1 are the weight and bias terms of the network that generates the convolution kernel, Activation is the activation function, and Conv is the convolution function.

[0118] After obtaining the first colony information and the second colony information, the number of colonies included in the image to be processed is determined according to the two colony information. Specifically, if there are no overlapping colonies in the first colony information and the second colony information, the sum of the first colony number and the second colony number is directly used as the number of colonies included in the image to be processed. If there are overlapping colonies in the first colony information and the second colony information, the overlapping colonies are determined based on the first colony position and the second colony position; the first colony number and the second colony number are deduplicated based on the overlapping colonies to obtain the number of colonies included in the image to be processed.

[0119] Specifically, the first segmented image and each second image are spliced ​​and the detection results are deduplicated by soft non-maximum suppression (Soft-NMS) to obtain the final counting result.

[0120] Among them, Soft-NMS is an improved non-maximum suppression (NMS) algorithm, which is used in target detection to reduce the target loss problem caused by hard threshold in standard NMS. Soft-NMS provides a more flexible way to handle overlapping detection frames, improving detection performance by gradually reducing the detection score instead of directly deleting the low-scoring frame.

[0121] The steps of standard NMS are as follows:

[0122] Select the detection box with the highest score: Select the box with the highest score (confidence) from all detection boxes as the benchmark.

[0123] Remove overlapping boxes: Remove all boxes whose overlap (IoU, intersection over union) with the reference box exceeds a certain threshold.

[0124] Repeat steps: Repeat the above steps until all boxes have been processed.

[0125] A major problem of standard NMS is the hard removal of overlapping boxes, which may lead to the incorrect removal of detection boxes that are close to the true objects but have slightly lower scores, especially in scenes with dense objects.

[0126] Soft-NMS proposes a method to gradually reduce the scores of overlapping detection boxes instead of directly removing these boxes. This allows detection boxes with high overlap but still potentially containing useful information to be retained, but only participate in the final result with a lower priority. The main steps of Soft-NMS are as follows:

[0127] Initialization: Same as standard NMS, all detection boxes are sorted according to the scores.

[0128] Select the detection box with the highest score: Select the detection box with the highest score as the reference box.

[0129] Score update: For each remaining detection box, update its score based on its overlap (IoU) with the reference box instead of removing it directly.

[0130] Specifically, Soft-NMS uses the intersection over union (IoU) to calculate the degree of overlap between candidate boxes, as shown in formula (13):

[0131]

[0132] Among them, B i and B j Represents two candidate boxes, namely, the bounding box of a bacterium in the first colony position and the bounding box of a bacterium in the second colony position, and Area represents the area of ​​the region.

[0133] For each candidate box B i , according to its and other candidate boxes B j The overlap adjustment score S i , the score adjustment formula of Soft-NMS is formula (14):

[0134]

[0135] Among them, σ is a hyperparameter used to control the strength of the adjustment; S i It is candidate box B i The original score.

[0136] To facilitate understanding of the overall implementation process of the technical solution provided by this application, see Figure 2 The framework diagram shown in Figure 2 As shown, the step includes:

[0137] (1) Multi-scale image segmentation

[0138] For each high-resolution culture dish image, the above segmentation method is used to obtain multiple small images with a size of 512x512.

[0139] (2) Colony segmentation

[0140] Segmentation: Each cut image is input into the trained U-Net model to obtain the colony segmentation mask. The goal of this step is to identify the colony area and generate preliminary segmentation results.

[0141] Segmentation screening: The preliminary segmentation results are divided into parts with large and small bacteria and / or colony adhesions and other parts.

[0142] For example Figure 3aThe structure diagram of the colony segmentation model shown in the figure processes the original image according to the predetermined segmentation method, inputs the colony segmentation model, and obtains the global colony segmentation mask. Among them, the Convolutional Block Attention Module (CBAM) is a simple and efficient forward convolutional neural network attention module. Given a feature map, CBAM infers the attention map along two separate dimensions, channel and spatial, and then multiplies the attention map with the input feature map for adaptive feature refinement. Because CBAM is a lightweight universal module, it can be seamlessly integrated into any CNN architecture with almost no impact on efficiency and computing power, and can achieve end-to-end training.

[0143] (3) Boundary segmentation

[0144] Boundary processing: The mask of large and small bacteria and / or colony adhesion obtained by U-Net segmentation is input into the boundary segmentation model (ESPNet combined with adaptive convolution), and the adherent colony area is refined to obtain a mask with precise boundaries.

[0145] For example, Figure 3b The schematic diagram of the colony boundary segmentation model structure is shown, and the mask of large and small bacteria and / or colony adhesion parts after colony segmentation is input into the colony boundary segmentation model to obtain a mask result with boundary segmentation.

[0146] (4) Detection and counting

[0147] Small image cutting: The image after boundary segmentation is reprocessed according to the multi-scale segmentation method to obtain a small image of size 512x512.

[0148] Detection processing: These small images are input into the improved yolov8 detection model to identify the size of the colonies in each small image. The detection results include the location and size information of the colonies.

[0149] Result stitching: stitch the detection results of each small image together with other parts of the preliminary segmentation result to restore the detection information of the entire image.

[0150] Deduplication processing: Use the Soft-NMS algorithm to deduplicate the test results, filter out the repeated detection parts, and obtain the final colony counting results.

[0151] like Figure 3cAs shown in the detection model structure diagram, the original image area corresponding to the boundary segmentation mask is cut to obtain a small image of size 512x512, and then input into the improved yolov8 detection model to obtain the detection result. The detection result of the small image is spliced ​​with the other parts after the colony segmentation, and the repeated detection parts are filtered through Soft-NMS to obtain the final counting result.

[0152] It can be seen that the technical solution provided by this application has the following characteristics:

[0153] Fast and accurate: By integrating the multi-scale Attention module and colony boundary segmentation, colony counting can be performed quickly and accurately, greatly improving processing efficiency.

[0154] Improve detection rate: The improved yolov8 applies fasterBlock in the feature extraction network and combines it with the Dysample dynamic upsampler to effectively improve the detection rate of colonies, especially when dealing with adherent colonies.

[0155] Practical significance: This method not only improves the accuracy of colony counting, but also shortens the experimental time. It has important practical application value in the fields of microbial research, clinical testing, and food safety, and provides strong technical support for researchers.

[0156] Based on the above method embodiment, the embodiment of the present application provides a colony counting device, which will be described below in conjunction with the accompanying drawings.

[0157] See also Figure 4 , which is a structural diagram of a colony counting device provided in an embodiment of the present application, such as Figure 4 As shown, the device 400 includes: an acquisition unit 401, a processing unit 402 and a detection unit 403.

[0158] The acquisition unit 401 is used to acquire the image to be processed, where the image to be processed is an image taken of a culture dish containing bacterial colonies;

[0159] A processing unit 402 is configured to perform multi-scale segmentation on the image to be processed to obtain a plurality of first images, wherein the multi-scale segmentation includes at least two different segmentation scales, and one first image corresponds to one segmentation scale;

[0160] The processing unit 402 is further configured to obtain a feature map corresponding to the first image corresponding to any segmentation scale, and perform weighted processing on the feature map based on an attention mechanism to obtain a first feature map;

[0161] The detection unit 403 is used to detect the colonies based on the first feature graphs corresponding to all segmentation scales to obtain the number of colonies.

[0162] In some embodiments, the processing unit 402 is specifically used to input the first image into a U-net network model, obtain a first feature map through the U-net network model, and the U-net network model integrates a multi-scale attention mechanism.

[0163] In some embodiments, the detection unit 403 is used to perform weighted fusion based on the first feature maps corresponding to all segmentation scales to obtain a target feature map; segment the image to be processed based on the target feature map to obtain a segmented image; and perform colony detection on the segmented image to obtain the number of colonies.

[0164] In some embodiments, if the colony includes large and small bacteria and / or colony adhesion areas, the detection unit 403 is specifically used to segment the image to be processed based on the target feature map to obtain a first segmented image and a second segmented image, wherein the first segmented image does not include large and small bacteria and colony adhesion areas, and the second segmented image includes the large and small bacteria and / or the colony adhesion areas, wherein the large and small bacteria refer to one bacterium containing one or more bacteria, and the colony adhesion area refers to an adhesion part between multiple bacteria; the colonies in the first segmented image are detected to obtain first colony information, wherein the first colony information includes the first colony number and the first colony position; the colonies in the second segmented image are boundary segmented to obtain a target image; the target image is segmented at multiple scales to obtain multiple second images, wherein one second image corresponds to one segmentation scale; for the second image corresponding to any segmentation scale, the second image is detected to obtain second colony information, wherein the second colony information includes the second colony position and the second colony number; the number of colonies included in the image to be processed is determined based on the first colony information and the second colony information.

[0165] In some embodiments, the detection unit 403 is specifically used to determine overlapping colonies based on the first colony position and the second colony position; and deduplicate the first colony number and the second colony number based on the overlapping colonies to obtain the number of colonies included in the image to be processed.

[0166] In some embodiments, the detection unit 403 is specifically used to perform colony detection on the segmented image using a yolov8 network model to obtain the number of colonies. The yolov8 network model includes an improved feature extraction network using FasterBlock and a Dysample dynamic upsampler.

[0167] In some embodiments, the image to be processed includes images taken under different light sources, and the feature map includes a feature map of the first image corresponding to the same segmentation scale of the different light sources.

[0168] It should be noted that the information execution process of each unit in the above-mentioned device and other contents can be specifically referred to the description in the method embodiment shown in the above-mentioned application, and will not be repeated here.

[0169] In addition, an embodiment of the present application provides an electronic device, including: a processor, a memory;

[0170] The memory is used to store computer-readable instructions or computer programs;

[0171] The processor is used to read the computer-readable instructions or the computer program so that the device implements the colony counting method.

[0172] An embodiment of the present application provides a computer-readable storage medium, including instructions or computer programs, which, when executed on a computer, enable the computer to execute the colony counting method described above.

[0173] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0174] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0175] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0176] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0177] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A colony counting method, characterized in that: The method comprises: Acquire an image to be processed, where the image to be processed is an image taken of a culture dish containing bacterial colonies; Performing multi-scale segmentation on the image to be processed to obtain a plurality of first images, wherein the multi-scale includes at least two different segmentation scales, and one first image corresponds to one segmentation scale; For a first image corresponding to any segmentation scale, a feature map corresponding to the first image is obtained, and a weighted process is performed on the feature map based on an attention mechanism to obtain a first feature map; The colonies are detected based on the first feature graphs corresponding to all segmentation scales to obtain the number of colonies, including: Perform weighted fusion based on the first feature maps corresponding to all segmentation scales to obtain the target feature map; Segmenting the image to be processed based on the target feature map to obtain a segmented image, including: If the colony includes large and small bacteria and / or a colony adhesion area, the image to be processed is segmented based on the target feature map to obtain a first segmented image and a second segmented image, wherein the first segmented image does not include large and small bacteria and the colony adhesion area, and the second segmented image includes the large and small bacteria and / or the colony adhesion area, wherein the large and small bacteria refer to one or more bacteria contained in one bacterium, and the colony adhesion area refers to an adhesion portion between multiple bacteria; Performing colony detection on the segmented image to obtain the number of colonies includes: Detecting the colonies in the first segmented image to obtain first colony information, where the first colony information includes the number of first colonies and first colony positions; Performing boundary segmentation on the bacterial colonies in the second segmented image to obtain a target image; Performing multi-scale segmentation on the target image to obtain a plurality of second images, where each second image corresponds to one segmentation scale; For a second image corresponding to any segmentation scale, detecting the second image to obtain second colony information, where the second colony information includes a second colony position and a second colony quantity; The number of colonies included in the to-be-processed image is determined based on the first colony information and the second colony information.

2. The method according to claim 1, characterized in that The step of obtaining a feature map corresponding to the first image and performing weighted processing on the feature map based on an attention mechanism to obtain a first feature map includes: The first image is input into a U-net network model, and a first feature map is obtained through the U-net network model, wherein the U-net network model integrates a multi-scale attention mechanism.

3. The method according to claim 1, characterized in that The determining the number of colonies included in the to-be-processed image based on the first colony information and the second colony information includes: determining overlapping colonies based on the first colony position and the second colony position; The first colony number and the second colony number are deduplicated based on the overlapping colonies to obtain the number of colonies included in the image to be processed.

4. The method according to any one of claims 1 to 3, characterized in that: The performing colony detection on the segmented image to obtain the number of colonies includes: The segmented image is subjected to colony detection using a yolov8 network model to obtain the number of colonies. The yolov8 network model includes a feature extraction network improved by FasterBlock and a Dysample dynamic upsampler.

5. The method according to claim 1, characterized in that The image to be processed includes images taken under different light sources, and the feature map includes a feature map of the first image corresponding to the same segmentation scale of the different light sources.

6. A colony counting device, characterized in that: The device comprises: An acquisition unit, used for acquiring an image to be processed, wherein the image to be processed is an image taken of a culture dish containing bacterial colonies; A processing unit, configured to perform multi-scale segmentation on the image to be processed to obtain a plurality of first images, wherein the multi-scale segmentation includes at least two different segmentation scales, and one first image corresponds to one segmentation scale; The processing unit is further configured to obtain, for a first image corresponding to any segmentation scale, a feature map corresponding to the first image, and perform weighted processing on the feature map based on an attention mechanism to obtain a first feature map; A detection unit, used to detect the colonies based on the first feature graphs corresponding to all segmentation scales to obtain the number of colonies; The detection unit is specifically used to perform weighted fusion based on the first feature maps corresponding to all segmentation scales to obtain a target feature map; if the colony includes large and small bacteria and / or colony adhesion areas, the image to be processed is segmented based on the target feature map to obtain a first segmented image and a second segmented image, the first segmented image does not include large and small bacteria and colony adhesion areas, the second segmented image includes the large and small bacteria and / or the colony adhesion areas, the large and small bacteria refer to one or more bacteria contained in one bacterium, and the colony adhesion area refers to the adhesion part between multiple bacteria; the colonies in the first segmented image are detected to obtain first colony information, the first colony information includes the first colony number and the first colony position; the colonies in the second segmented image are segmented to obtain a target image; the target image is segmented at multiple scales to obtain multiple second images, one second image corresponds to one segmentation scale; for the second image corresponding to any segmentation scale, the second image is detected to obtain second colony information, the second colony information includes the second colony position and the second colony number; the number of colonies included in the image to be processed is determined based on the first colony information and the second colony information.

7. An electronic device, characterized in that: Including: processor, memory; The memory is used to store computer-readable instructions or computer programs; The processor is used to read the computer-readable instructions or the computer program so that the electronic device implements the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a device, the device executes the method according to any one of claims 1 to 5.

Images