A method and apparatus for counting microorganisms using a composite image

Abstract

The application discloses a kind of method and device for microorganism counting using combined image, it is related to microorganism counting technical field, including, using convolutional neural network model to extract the core feature point of preprocessed microscopic image, and the similarity between core feature point is calculated, obtain the microorganism microscopic image containing splicing position.Utilize adaptive splicing algorithm to the splicing of microorganism microscopic image, identify and correct the error that appears, obtain the panoramic image after splicing.Cut the panoramic image after splicing into graph algorithm, construct the energy function of panoramic image, find out the lowest energy segmentation path using minimum cut algorithm, generate the panoramic image of microorganism region.The application greatly improves the matching precision between adjacent images and the stability of geometric transformation, and the use of breadth-first search algorithm and minimum cut algorithm improves the segmentation accuracy of image segmentation, avoids the situation that image is misaligned and overlaps inconsistently.

Description

Technical Field

[0001] The present invention relates to the technical field of microorganism counting, and in particular to a method and device for counting microorganisms using combined images. Background Art

[0002] Microbial counting is a crucial analytical step in modern biomedical research, environmental monitoring, and other environments. In recent years, with advances in digital image processing, automated counting methods based on computer vision have become mainstream, primarily focusing on object detection and segmentation in single-frame microscopic images.

[0003] Existing microbial counting methods also face various problems. Many conventional stitching algorithms rely on single feature descriptors (such as SIFT feature points), which can easily lead to mismatches in areas with dense microbial populations. Furthermore, the geometric transformation matrix solution often ignores the correction step, resulting in image misalignment or artifacts. While conventional segmentation algorithms using graph cuts can incorporate energy functions, they are computationally complex and lack integration of features like color and texture, often resulting in over-segmentation or under-segmentation. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for microbial counting using combined images to solve the problem of feature matching deviation in existing microbial counting technologies.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for counting microorganisms using combined images, comprising collecting microscopic images of microorganisms and performing preprocessing;

[0008] A convolutional neural network model is used to extract the core feature points of the preprocessed microscopic image and calculate the similarity between the core feature points to obtain a microbial microscopic image containing the splicing position;

[0009] Adaptive stitching algorithm is used to stitch microbial microscopic images. During the stitching process, errors are identified and corrected to obtain a stitched panoramic image.

[0010] The stitched panoramic image is input into the graph cut algorithm to construct the energy function of the panoramic image. The minimum cut algorithm is used to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area.

[0011] performing morphological operations on the panoramic image of the microbial area, optimizing the boundary of the microbial area, and outputting the optimized panoramic image;

[0012] Microorganism counting is performed based on the optimized panoramic image, and a microorganism counting result report is output.

[0013] As a preferred solution of the method for counting microorganisms using combined images described in the present invention, the preprocessing includes grayscale processing, equalization processing, noise removal and edge detection.

[0014] As a preferred embodiment of the method for counting microorganisms using combined images of the present invention, a convolutional neural network model is used to extract core feature points of the preprocessed microscopic image, and the similarity between the core feature points is calculated to obtain a microbial microscopic image containing the splicing position, comprising the following steps:

[0015] Select the CNN model, input the preprocessed microscopic image into the CNN model, and output a set of feature maps;

[0016] Perform global pooling on the feature map set to obtain the feature vector set of the microbial microscopic image and generate feature point descriptors, then match the feature point descriptors and output the feature point matching results;

[0017] According to the feature point matching results, the geometric transformation matrix between adjacent images is output and applied to each pair of adjacent images to obtain a microbial microscopic image containing the splicing position.

[0018] As a preferred embodiment of the method for counting microorganisms using combined images described in the present invention, an adaptive stitching algorithm is used to perform feathering processing, color correction, stitching error detection, and stitching error correction on the microbial microscopic image, and a stitched panoramic image is output.

[0019] As a preferred embodiment of the method for microbial counting using combined images of the present invention, the stitched panoramic image is input into a graph cut algorithm, an energy function of the panoramic image is constructed, and a minimum cut algorithm is used to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area, including the following steps:

[0020] Construct an energy function for the panoramic image, treat each pixel of the stitched panoramic image as a graph node, and treat the adjacent pixels of each pixel as edges to construct a graph structure;

[0021] The weights of the edges in the graph structure are calculated, and based on the energy function of the panoramic image, the segmentation path with the lowest energy is found and assigned a unique identifier, and a panoramic image of the microbial area is output.

[0022] As a preferred embodiment of the method for counting microorganisms using combined images of the present invention, the method includes the following steps: performing morphological operations on the panoramic image of the microorganism area, optimizing the boundary of the microorganism area, and outputting the optimized panoramic image.

[0023] Performing morphological operations, selecting structural elements and repeatedly scanning each pixel of the panoramic image of the microbial area, and outputting the panoramic image after the morphological operation;

[0024] The U-Net model is selected as the deep learning model, and the panoramic image after morphological operation is input into the U-Net model to obtain the optimized panoramic image.

[0025] As a preferred embodiment of the method for performing microbial counting using a combined image according to the present invention, the method comprises the following steps: performing microbial counting based on the optimized panoramic image and outputting a microbial counting result report.

[0026] Select the classification standard of microorganisms to classify the microbial areas, traverse each microbial area, count the number of microorganisms and summarize them to obtain the microbial statistical results;

[0027] Consolidate microbial statistics results into a microbial count results report.

[0028] In a second aspect, the present invention provides a device for counting microorganisms using combined images, comprising: an image acquisition module for acquiring microscopic images of microorganisms and performing preprocessing;

[0029] The feature extraction module uses a convolutional neural network model to extract the core feature points of the preprocessed microscopic image and calculates the similarity between the core feature points to obtain a microbial microscopic image including the splicing position;

[0030] The image stitching module uses an adaptive stitching algorithm to stitch microbial microscopic images. During the stitching process, it identifies and corrects errors to obtain a stitched panoramic image.

[0031] The image generation module inputs the stitched panoramic image into the graph cut algorithm, constructs the energy function of the panoramic image, and uses the minimum cut algorithm to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area;

[0032] An image optimization module performs morphological operations on the panoramic image of the microbial area, optimizes the boundary of the microbial area, and outputs the optimized panoramic image;

[0033] The report generation module performs microbial counts based on the optimized panoramic image and outputs a microbial count result report.

[0034] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for counting microorganisms using combined images as described in the first aspect of the present invention is implemented.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for counting microorganisms using combined images as described in the first aspect of the present invention is implemented.

[0036] The beneficial effects of the present invention are as follows: a convolutional neural network model is used to extract the core feature points of the preprocessed microscopic image, and the similarity between the core feature points is calculated to obtain a microbial microscopic image containing the stitching position. The microbial microscopic images are stitched using an adaptive stitching algorithm. During the stitching process, the errors that occur are identified and corrected to obtain a stitched panoramic image. The application of the FLANN and RANSAC algorithms greatly improves the matching accuracy between adjacent images and the stability of geometric transformations, while the feathering technology and color correction significantly improve the transition effect and smoothness of the overlapping areas, making the stitched panoramic image more natural and consistent. The use of the breadth-first search algorithm and the minimum cut algorithm improves the segmentation accuracy of image segmentation and avoids the image dislocation and overlapping inconsistencies that occur in conventional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is an overall flow chart of the microbial counting method in Example 1.

[0039] Figure 2 Schematic diagram of image stitching based on FLANN-RANSAC in Example 1.

[0040] Figure 3 This is a schematic diagram of the segmentation of the minimum cut algorithm in Example 1. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0044] Example 1, reference Figures 1 to 3 , which is the first embodiment of the present invention, provides a method for counting microorganisms using a combined image, comprising the following steps:

[0045] S1. Collect microbial microscopic images and perform preprocessing.

[0046] The following steps are included:

[0047] The grayscale function in ImageJ image processing software is used to convert the color value of each pixel in the microbial microscopic image into a grayscale value, and the grayscale microbial microscopic image is output;

[0048] The histogram equalization function in the ImageJ image processing software is used to perform histogram equalization on the grayscale microbial microscopic image, and the equalized microbial microscopic image is output;

[0049] Use the Gaussian filter kernel of the Gaussian filter to remove the noise of the microbial microscopic image after equalization processing, and output the denoised microbial microscopic image;

[0050] The Canny edge detection algorithm is used to perform edge detection on the denoised microbial microscopic image, and the microbial microscopic image after edge detection is output.

[0051] S2. A convolutional neural network model is used to extract the core feature points of the preprocessed microscopic image and calculate the similarity between the core feature points to obtain a microbial microscopic image containing the splicing positions.

[0052] The following steps are included:

[0053] S2.1. Select the CNN model (i.e., convolutional neural network model) as the feature extractor, and use the sliding window technology to divide the preprocessed microbial microscopic image into several image blocks of the same size (e.g., each image block is 256x256 pixels in size), and then input each image block into the CNN model one by one. The CNN model will automatically generate a feature map for each image block. The feature map is a multi-channel matrix output by the convolution layer, and each channel represents a feature pattern. Specifically, the CNN model uses convolutional layer operations and ReLU activation functions to extract the core features of the image block (including the edge contour, texture, color, intensity distribution, and local pattern structure of the image block. The shallow convolution layer extracts low-level features such as edges and textures, and the deep convolution layer extracts high-level features such as local pattern structures) and combines the core features of each image block into a feature map set;

[0054] Next, a global pooling operation is performed on the feature map collection. Specifically, for each feature map, the average value of all pixels in each channel is calculated and combined into a feature vector of uniform length. Each feature vector is then concatenated sequentially according to the position of the image blocks in the original image to form a feature vector collection for the entire microbial microscopic image.

[0055] S2.2. Apply L2 normalization to each feature vector in the feature vector set of the microbial microscopic image to generate a feature point descriptor. Specifically, divide each feature vector by its own Euclidean norm and adjust the modulus of each feature vector to 1 to obtain a normalized feature vector (i.e., feature point descriptor).

[0056] S2.3. Determine the overlap between adjacent images. Specifically, this is determined based on the microscope settings and the order in which the images were taken. For example, if an automated stage is used for continuous imaging, the overlap between adjacent images can be estimated based on the distance traveled during each stage (e.g., the step length of the stepper motor). Typically, the overlap between adjacent images accounts for 20% to 30% of the total image area.

[0057] S2.4. Match the feature point descriptors in adjacent images based on their overlapping regions. Specifically, the FLANN nearest neighbor matching algorithm uses a fast approximate search (with the search parameter "checks") to sequentially compare the feature point descriptors in each pair of adjacent images, finding the most similar matching point pair among all feature points. The similarity (i.e., Euclidean distance) between each pair of matching points is then calculated, and the matching point with the smallest Euclidean distance is selected as the matching result (i.e., the feature point matching result).

[0058] S2.5. Based on the matching results, calculate the geometric transformation matrix between adjacent images. Specifically, select the Random Sample Consensus (RANSAC) algorithm as the matching algorithm, then set the number of RANSAC algorithm iterations (1000), the inlier threshold (i.e., a fixed pixel distance, such as 5 pixels), and the minimum number of samples (4). Then, randomly select a set of points from the matching point pairs and select the minimum number of samples as the initial sample point set (each sample point contains two coordinates, one on the first image and one on the second image. Assume that the coordinates of sample point A are (x1, y1) and the coordinates of sample point B are (x2, y2)). Calculate the homography matrix for the initial sample point set. The calculation process is as follows: the relationship between the coordinates of each sample point is recorded to form a correspondence (for example, the correspondence between the coordinates of sample point A and sample point B is (x1, y1) -> (x2, y2)). Using translation, rotation, and scaling operations, the sample point on (x1, y1) is mapped to the sample point on (x2, y2). After the mapping is completed, a system of equations is constructed to align all the mapped sample points (this system of equations can be understood as describing the correspondence between all mapped sample points). The transformation parameters of the system of equations (i.e., the homography matrix values) are then solved and the coordinates of each sample point on the first image are transformed using the homography matrix to obtain the new coordinates on the second image. In this way, all sample points can be mapped according to the correspondence.

[0059] S2.6. Based on the homography matrix of the initial sample point set, calculate the distance between all mapped sample points and the actual sample points. If the distance is less than the inlier threshold value preset by the RANSAC algorithm, the matching point pair is considered an inlier, that is, a point pair that meets the current transformation matrix (which can be understood as an error assessment of the mapping). Then, count the specific number of inliers and update the homography matrix. Repeat the three operations of selecting a sample point set, calculating the homography matrix, and counting the number of inliers until the set number of iterations is reached. Then, select the homography matrix with the most inliers as the final geometric transformation matrix.

[0060] S2.7. Apply the geometric transformation matrix to each pair of adjacent images, count the size of the overlapping area and the translation amount of each pair of adjacent images, and output the microbial microscopic image containing the stitching position.

[0061] S3. Use an adaptive stitching algorithm to stitch the microbial microscopic images. During the stitching process, identify and correct any errors that occur to obtain a stitched panoramic image.

[0062] The following steps are included:

[0063] S3.1. First, select an image as a reference image, and then perform the image fusion process of the microbial microscopic images. Specifically, a feathering technique is used to set a transition zone (width of 20-30 pixels) around the overlapping area of ​​each pair of adjacent images. Then, a Gaussian weighting method is used to weighted average all pixels in each pair of adjacent images, making the gradient transition of the overlapping area smoother and more natural. Specifically, all pixels in each pair of adjacent images are assigned a weight based on a Gaussian distribution. The closer the pixel is to the center of the overlapping area, the higher the weight, and vice versa.

[0064] S3.2. Calculate the color mean and standard deviation for each pair of adjacent images to determine the overall brightness and contrast of each pair of adjacent images. Based on the overall brightness and contrast, evenly adjust the color tones of each pair of adjacent images. Specifically, use histogram matching to adjust the color histogram of each pair of adjacent images to be consistent with the reference image. Histogram matching remaps the pixel values ​​of each pair of adjacent images to achieve a uniform color distribution between the two images. For example, if one image in a pair of adjacent images is darker and the other is lighter, adjust the pixel values ​​of the adjacent images to make their color distributions more consistent.

[0065] S3.3. During the stitching process, problems such as image misalignment or inconsistent overlap may occur. The accuracy of the stitching results needs to be verified using the matching results of feature points. This check ensures that the positions of all matched feature points remain consistent in the stitched image. Set an error threshold (specifically, the range is 1 to 5 pixels, which can be adjusted based on the actual stitching situation). If the distance between the actual position of a feature point and the expected position exceeds 5 pixels, the feature point is considered to have a stitching error.

[0066] S3.4. After detecting stitching errors, use a local optimization algorithm to fine-tune the geometric transformation matrix. This involves identifying feature points with large errors and recalculating the local transformation matrix. For example, several feature points with large errors can be selected as a new sample point set. The geometric transformation matrix can then be recalculated and updated, and then iterated again according to a set number of iterations (i.e., the geometric transformation matrix is ​​continuously adjusted to gradually reduce the stitching error until the stitching errors for all feature points are within the error threshold).

[0067] S3.5. Merge the images after feathering, color correction, and stitching error correction into a complete panoramic image and save it in a standard format (TIFF) to generate a stitched panoramic image.

[0068] S4. Input the stitched panoramic image into the graph cut algorithm, construct the energy function of the panoramic image, use the minimum cut algorithm to find the segmentation path with the lowest energy, and generate a panoramic image of the microbial area.

[0069] The following steps are included:

[0070] S4.1. Construct an energy function for the panoramic image based on a data term (determined by the color or brightness of the microorganisms under illumination) and a smoothness term. Specifically, the data term measures the probability of each pixel in the stitched panoramic image belonging to a specific region. This means that the microorganism region is determined based on the grayscale value or color distribution of each pixel in the stitched panoramic image. The smoothness term, on the other hand, measures the similarity between adjacent pixels in the stitched panoramic image. This is calculated by calculating the grayscale difference between adjacent pixels in the stitched panoramic image and serves as part of the energy function. The specific calculation process is as follows: Since each pixel has four direct neighbors (i.e., above, below, left, and right), the calculation can be divided into two cases. For grayscale images, the grayscale difference between adjacent pixels in the stitched panoramic image can be determined by comparing the grayscale values ​​of the two pixels. For example, if a pixel has a grayscale value of 50 and its upper neighbor has a grayscale value of 55, the difference between the two pixels is 5. For color images, the difference is calculated for each color channel (red, green, and blue). For example, if the RGB value of a pixel is (100, 150, 200) and the RGB value of the adjacent pixel on the right is (105, 153, 198), the differences between the three color channels are 5, 3, and 2 respectively.

[0071] S4.2. Accumulate the grayscale differences between adjacent pixels of all stitched panoramic images to obtain a total difference value. Assign a weight to the total difference value based on the actual segmentation requirements, and calculate the smoothness term contribution value (for example, if the weight of the smoothness term is set to 0.2 and the total difference value is 3.5, then the smoothness term contribution value is 3.5 × 0.2 = 0.7). Integrate the smoothness term contribution value into the energy function to form a part of the energy function (assuming there are 100 pixels and the smoothness term contribution value of each pixel is 0.7, then the total smoothness term energy is 100 × 0.7 = 70).

[0072] S4.3. Consider each pixel of the stitched panoramic image as a graph node and its neighboring pixels as edges to construct a graph structure. Two special nodes are set: a source node and a sink node. The source node represents the foreground area, and the sink node represents the background area. Each edge connects to the source node and the sink node.

[0073] Based on the actual requirements of the smoothness term, the edge weights between each pixel and its neighbors are determined (the smaller the grayscale value, the larger the edge weight, indicating a higher probability that the two pixels belong to the same region). Then, based on the actual requirements of the data term, the edge weights between each pixel and the source and sink are determined. Specifically, the probability of each pixel belonging to the foreground or background region is determined based on pixel characteristics such as color and texture. For example, if a pixel's color is closer to that of a microorganism, the edge weight between it and the source is larger; conversely, if a pixel is closer to the background color, the edge weight between it and the sink is larger.

[0074] S4.4. Use the minimum splitting algorithm to find the lowest-energy segmentation path based on the constructed energy function. The specific steps are to assign initial weights to all edges in the graph structure and initialize the state of the flow network (a flow network is a directed graph used to help achieve effective image segmentation). Then, use the breadth-first search algorithm to search for an augmenting path from the source to the sink (the flow of all edges is set to 0 at the beginning of the search). During the search, record the residual capacity of each edge (i.e., the maximum flow that the edge can carry). If the current flow of a forward edge is less than the residual capacity, it means that the forward edge has positive residual capacity. For reverse edges, if the current flow of the reverse edge is less than the residual capacity, it means that the reverse edge has flow that can be backtracked.

[0075] Based on the residual capacity of all edges, we check whether each edge along the search path has sufficient residual capacity. Specifically, if the edge from the current node to the next node has a positive residual capacity, we continue to move forward; otherwise, we must backtrack or stop the search. In this way, we gradually build an augmenting path from the source to the sink, where each edge has a positive residual capacity.

[0076] S4.5. Calculate the minimum residual capacity of the augmenting path from the source to the sink, and update the actual flow of each edge on the augmenting path based on the minimum residual capacity. (For forward edges, increase the actual flow and decrease the residual capacity. For reverse edges, increase the actual flow by the same amount as the forward edge while decreasing the residual capacity. This can be thought of as using reverse edges to backtrack the actual flow.) After the update is complete, continue searching from the source. Each time a new augmenting path is found, update the state of the flow network and recalculate the residual capacity until no more augmenting paths can be found from the source to the sink. When there are no more augmenting paths, the maximum flow (i.e., the minimum cut) has been found. This minimum cut is the path with the lowest energy.

[0077] S4.6. Save the segmentation path with the lowest energy as an image file in PNG format, assign a unique identifier (such as a color label) to each microbial area in the segmentation path, and mark it in the image file to form a panoramic image of the microbial area.

[0078] The panoramic image of the microbial area obtained in this step constructs a more comprehensive energy function by simultaneously considering both the data term and the smoothness term. Compared to traditional image segmentation methods that may rely solely on a single feature (such as color or texture), this not only improves segmentation accuracy but also ensures the smoothness and continuity of the segmentation results. By finding augmenting paths through a breadth-first search algorithm and gradually approaching the maximum flow by updating the flow network state, the algorithm's efficiency is greatly improved while ensuring the quality of the segmentation results.

[0079] S5. Perform morphological operations on the panoramic image of the microbial region, optimize the boundary of the microbial region, and output the optimized panoramic image.

[0080] The following steps are included:

[0081] S5.1 Morphological operations include dilation (filling small holes within the microbial region) and erosion (removing isolated small noise points). Taking dilation as an example, first select a suitable structuring element (such as a 3x3 square). Starting from the upper left corner of the panoramic image of the microbial region, use the structuring element to scan each pixel row and column by row (the current pixel position of the panoramic image of the microbial region is set as the center point, and the structuring element is applied to the center point and the area near the center point. Then, all pixels covered by the structuring element are checked one by one to see if any of them belong to the microbial region. If so, the center point is marked as part of the microbial region, even if the center point does not originally belong to the microbial region).

[0082] After repeatedly scanning each pixel until the panoramic image of the entire microbial area is traversed, the small holes in the microbial area can be effectively filled in this way, making the microbial area more complete and continuous.

[0083] S5.2. Select the U-Net model (U-shaped network model) as the deep learning model. Input the panoramic image after morphological manipulation into the U-Net model. The U-Net model gradually restores the resolution of the panoramic image after morphological manipulation through the decoder and generates pixel-level classification results (i.e., the probability of each pixel belonging to a microbial region). For each identified microbial region, the U-Net model further optimizes the boundaries of the microbial region. Specifically, the U-Net model adjusts the boundaries of the microbial region to make them clearer and more accurate based on edge information (such as gradient changes) and texture features (such as local consistency) in the panoramic image after morphological manipulation. For example, the U-Net model adds more details (such as small protrusions and depressions) near the boundaries of the microbial region or smooths discontinuities (i.e., interruptions or unevenness on the boundaries of the microbial region, such as jagged edges and broken lines) to improve segmentation accuracy.

[0084] The U-Net model consists of an encoder (for feature extraction) and a decoder (for resolution restoration). The encoder consists of multiple convolutional and pooling layers, while the decoder includes upsampling and convolutional layers. The U-Net model is trained as follows: a panoramic image of a microbial region is selected and a mini-batch of panoramic images and labels is extracted as the training set. This training set is fed into the U-Net encoder, which progressively reduces the spatial resolution of the panoramic image through a series of convolutional and pooling layers while increasing the number of channels to capture more complex features. Each layer generates a panoramic feature map representing different levels of abstract information. All panoramic feature maps output by the encoder then enter the decoder. The decoder gradually restores the spatial resolution of the original panoramic image through deconvolution and upsampling operations. During this process, the decoder incorporates skip connections from the corresponding encoder layers to preserve panoramic details and improve segmentation accuracy. Ultimately, the decoder outputs a probability map of the same size as the input microbial region panoramic image, where each pixel value represents the probability of belonging to a microbial region.

[0085] The output of the decoder is converted into a binary prediction, which is then compared with the true label (i.e., the label information of whether each pixel belongs to the microbial area). The average loss value of the entire batch of panoramic images is calculated (i.e., the loss function uses binary cross entropy loss. For each pixel, the logarithmic loss between the predicted probability and the actual label is calculated, and the average of all pixels is taken as the final average loss value).

[0086] Using the autograd functionality in PyTorch's automatic differentiation tool, we compute the gradients (i.e., backpropagation) of all learnable parameters (weights and biases) of the U-Net model based on the average loss across the entire batch of panoramic images. We then use the Adam optimization algorithm to update the U-Net model's network parameters based on the computed gradients. Specifically, the Adam optimization algorithm adjusts the small step size (learning rate) and momentum for each parameter to minimize loss while avoiding oscillation or getting stuck in local minima.

[0087] Set an appropriate number of iterations (such as 100) and repeat the forward propagation, loss calculation, and backpropagation operations until the set number of iterations is reached or the U-Net model converges. The U-Net model training is completed.

[0088] In this step, the U-Net model is used to optimize the boundaries of the microbial area using edge information and texture features. Compared with existing segmentation methods that result in less smooth and natural segmentation results when processing target boundaries, this method significantly improves the accuracy and visual effect of boundary segmentation.

[0089] S6. Perform microbial counting based on the optimized panoramic image and output a microbial counting result report.

[0090] The following steps are included:

[0091] S6.1. Based on the optimized panoramic image and actual needs, select appropriate microbial classification criteria to distinguish different types of microorganisms. For example, microorganisms can be classified based on their shape into spheres, rods, or other complex shapes, or based on their color into red, green, or other different categories.

[0092] S6.2. Classify each microbial region according to the microbial classification criteria. Specifically, compare the shape, color, and texture of each microorganism with the microbial classification criteria. For example, if shape is used as the classification criterion, the shape type of the microorganism (e.g., rectangular, elliptical, irregular, etc.) can be determined by calculating the ratio of the perimeter to the area of ​​each microbial region.

[0093] S6.3. Initialize a counter for each microorganism category. For example, if microorganisms are classified into three categories (spherical, rod-shaped, and other shapes), initialize three counters. Then, iterate through each microorganism region one by one, updating the corresponding counter based on the classification result of the microorganism region. The specific method is to examine the characteristics of each microorganism region and classify the microorganism into the corresponding category based on the microorganism classification criteria. For example, if a microorganism region is identified as spherical, increment the value of the spherical category counter.

[0094] S6.4. After counting the number of microorganisms, the number of microorganisms in each category needs to be summarized. This is done by summing the counter values ​​for each category to obtain the final microbial count. For example, after counting, there are 50 spherical microorganisms, 30 rod-shaped microorganisms, and 15 microorganisms of other shapes. The final microbial count results are combined into a text file to form a microbial count report (including the microbial classification criteria, the microbial classification results, and the total number of microorganisms).

[0095] This embodiment also provides a device for counting microorganisms using combined images, comprising: an image acquisition module for acquiring microscopic images of microorganisms and performing preprocessing;

[0096] The feature extraction module uses a convolutional neural network model to extract the core feature points of the preprocessed microscopic image and calculates the similarity between the core feature points to obtain a microbial microscopic image including the splicing position;

[0097] The image stitching module uses an adaptive stitching algorithm to stitch microbial microscopic images. During the stitching process, it identifies and corrects errors to obtain a stitched panoramic image.

[0098] The image generation module inputs the stitched panoramic image into the graph cut algorithm, constructs the energy function of the panoramic image, and uses the minimum cut algorithm to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area;

[0099] An image optimization module performs morphological operations on the panoramic image of the microbial area, optimizes the boundary of the microbial area, and outputs the optimized panoramic image;

[0100] The report generation module performs microbial counts based on the optimized panoramic image and outputs a microbial count result report.

[0101] This embodiment also provides a computer device suitable for the method of using combined images to count microorganisms, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the method of using combined images to count microorganisms as proposed in the above embodiment.

[0102] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0103] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for implementing microbial counting using combined images as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0104] In summary, the present invention obtains a microbial microscopic image containing a stitching position by: using a convolutional neural network model to extract the core feature points of the preprocessed microscopic image, and calculating the similarity between the core feature points. The microbial microscopic image is stitched using an adaptive stitching algorithm, and during the stitching process, the errors that occur are identified and corrected to obtain a stitched panoramic image. The application of FLANN and RANSAC algorithms greatly improves the matching accuracy between adjacent images and the stability of geometric transformations, while feathering technology and color correction significantly improve the transition effect and smoothness of overlapping areas, making the stitched panoramic image more natural and consistent. The use of breadth-first search algorithm and minimum cut algorithm improves the segmentation accuracy of image segmentation and avoids the image dislocation and overlapping inconsistencies that occur in conventional methods.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for counting microorganisms using combined images, characterized in that: include, Collect microbial microscopic images and perform preprocessing; A convolutional neural network model is used to extract the core feature points of the preprocessed microscopic image and calculate the similarity between the core feature points to obtain a microbial microscopic image containing the splicing position; Adaptive stitching algorithm is used to stitch microbial microscopic images. During the stitching process, errors are identified and corrected to obtain a stitched panoramic image. The stitched panoramic image is input into the graph cut algorithm to construct the energy function of the panoramic image. The minimum cut algorithm is used to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area. The process includes the following steps: Constructing the energy function of the panoramic image based on the data term and the smooth term; The grayscale differences between adjacent pixels of all stitched panoramic images are accumulated to obtain a total difference value. According to the actual segmentation requirements, a weight is assigned to the total difference value, and the smoothness contribution value is calculated. The smoothness contribution value is integrated into the energy function to form a part of the energy function. Each pixel of the stitched panoramic image is considered as a graph node, and the adjacent pixels of each pixel are considered as edges. A graph structure is constructed, and two special nodes are set: one is the source node and the other is the sink node. The source node represents the foreground area, and the sink node represents the background area. Each edge is connected to the source node and the sink node. Using the minimum segmentation algorithm, we can find the segmentation path with the lowest energy based on the constructed energy function. Calculate the minimum residual capacity of the augmenting path from the source to the sink, and update the actual flow of each edge on the augmenting path based on the minimum residual capacity. After the update is completed, continue searching from the source. Each time a new augmenting path is found, update the state of the flow network and recalculate the residual capacity until no more augmenting paths can be found from the source to the sink. When there are no more augmenting paths, it proves that the maximum flow has been found. The maximum flow is the segmentation path with the lowest energy. The segmentation path with the lowest energy is saved in an image file, and a unique identifier is assigned to each microbial area in the segmentation path, and marked in the image file to form a panoramic image of the microbial area; performing morphological operations on the panoramic image of the microbial area, optimizing the boundary of the microbial area, and outputting the optimized panoramic image; Microorganism counting is performed based on the optimized panoramic image, and a microorganism counting result report is output.

2. The method for counting microorganisms using combined images according to claim 1, wherein: The preprocessing includes grayscale processing, equalization processing, noise removal and edge detection.

3. The method for counting microorganisms using combined images according to claim 2, wherein: The convolutional neural network model is used to extract the core feature points of the preprocessed microscopic image and calculate the similarity between the core feature points to obtain the microbial microscopic image containing the splicing position, including the following steps: Select the CNN model, input the preprocessed microscopic image into the CNN model, and output a set of feature maps; Perform global pooling on the feature map set to obtain the feature vector set of the microbial microscopic image and generate feature point descriptors, then match the feature point descriptors and output the feature point matching results; According to the feature point matching results, the geometric transformation matrix between adjacent images is output and applied to each pair of adjacent images to obtain the microbial microscopic image containing the splicing position.

4. The method for counting microorganisms using combined images according to claim 3, wherein: An adaptive stitching algorithm is used to feather the microbial microscopic image, perform color correction, detect stitching errors, and correct stitching errors, and output a stitched panoramic image.

5. The method for counting microorganisms using combined images according to claim 4, wherein: The stitched panoramic image is input into the graph cut algorithm to construct the energy function of the panoramic image. The minimum cut algorithm is used to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area. The process includes the following steps: Construct an energy function for the panoramic image, treat each pixel of the stitched panoramic image as a graph node, and treat the adjacent pixels of each pixel as edges to construct a graph structure; The weights of the edges in the graph structure are calculated, and based on the energy function of the panoramic image, the segmentation path with the lowest energy is found and assigned a unique identifier, and a panoramic image of the microbial area is output.

6. The method for counting microorganisms using combined images according to claim 5, wherein: Performing morphological operations on the panoramic image of the microbial area, optimizing the boundary of the microbial area, and outputting the optimized panoramic image includes the following steps: Performing morphological operations, selecting structural elements and repeatedly scanning each pixel of the panoramic image of the microbial area, and outputting the panoramic image after the morphological operation; The U-Net model is selected as the deep learning model, and the panoramic image after morphological operation is input into the U-Net model to obtain the optimized panoramic image.

7. The method for counting microorganisms using combined images according to claim 6, wherein: Perform microbial counting based on the optimized panoramic image and output a microbial counting result report, including the following steps: Select the classification standard of microorganisms to classify the microbial areas, traverse each microbial area, count the number of microorganisms and summarize them to obtain the microbial statistical results; Consolidate microbial statistics results into a microbial count results report.

8. A device for counting microorganisms using combined images, based on the method for counting microorganisms using combined images according to any one of claims 1 to 7, characterized in that: include, Image acquisition module, which collects microbial microscopic images and performs preprocessing; The feature extraction module uses a convolutional neural network model to extract the core feature points of the preprocessed microscopic image and calculates the similarity between the core feature points to obtain a microbial microscopic image including the splicing position; The image stitching module uses an adaptive stitching algorithm to stitch microbial microscopic images. During the stitching process, it identifies and corrects errors to obtain a stitched panoramic image. The image generation module inputs the stitched panoramic image into the graph cut algorithm, constructs the energy function of the panoramic image, and uses the minimum cut algorithm to find the segmentation path with the lowest energy to generate a panoramic image of the microbial area; An image optimization module performs morphological operations on the panoramic image of the microbial area, optimizes the boundary of the microbial area, and outputs the optimized panoramic image; The report generation module performs microbial counts based on the optimized panoramic image and outputs a microbial count result report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for counting microorganisms using combined images according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for counting microorganisms using combined images according to any one of claims 1 to 7 are implemented.

Images