Phytoplankton microscopic image recognition and cell counting method based on multi-task learning

Abstract

The present invention discloses a phytoplankton microscopic image recognition and cell counting method based on multi-task learning, which belongs to the field of resources and environment. ResNet50 is used as the backbone network to extract the deep features of the microscopic images of algae colony cells, and the deep features are respectively input into two parallel processing branches: the classification branch and the counting branch. The classification branch outputs the algae species recognition results of the microscopic image through a fully connected layer and a Softmax layer; the counting branch introduces a multi-scale feature fusion module to improve the model's robustness to the morphological differences of different single cells and cell overlap, constructs a regression model for algae cell density estimation, and counts the number of algae cells. The model training adopts a multi-task learning strategy, and its loss function consists of two parts: the cross entropy loss for algae species recognition and the mean square error loss for cell density map estimation. The model parameters are updated through an alternating training method, achieving accurate recognition of phytoplankton species and accurate counting of colony cells.

Description

Technical Field

[0001] The present invention belongs to the field of resources and environment, and specifically relates to a phytoplankton microscopic image recognition and cell counting method based on multi-task learning. Background Art

[0002] Phytoplankton are indicator organisms of the aquatic ecological environment, and their species distribution and cell abundance are important indicators for the evaluation of the aquatic ecological environment. At present, phytoplankton identification is still mainly based on manual microscopic examination, which cannot meet the current large-scale and high-frequency monitoring needs of aquatic organisms. In recent years, image-based automatic phytoplankton identification technology has developed rapidly, and a large number of research results have been achieved in the identification of phytoplankton species. However, phytoplankton in natural water bodies often form groups through cell division, cell adhesion, and other pathways, and the cell morphology of the group varies. For example, the multicellular structure of the phytoplankton is arranged in a disc or star-shaped structure, the multicellular structure of the phytoplankton is arranged in a flat or grid-like structure, and the multicellular structure of the Microcystis is arranged in a group of cells in an irregular manner. Existing methods for counting phytoplankton groups mainly use the area method to estimate the number of cells. However, due to the large differences in area between different cells of the same phytoplankton species and the existence of problems such as cell occlusion, the simple counting method based on the area method has large errors. In addition, some scholars have tried to customize the colony cell counting method based on the morphological characteristics of phytoplankton. However, there are many types of phytoplankton, and artificially customizing the colony cell counting method for each type of phytoplankton colony cell has defects such as high R&D costs and insufficient generalization ability for counting across algae species. Summary of the Invention

[0003] To address the problem that phytoplankton colonies in natural water bodies are difficult to accurately count due to their variable morphology, overlapping cell adhesions, and significant inter-species differences, the present invention discloses an efficient, robust, and universal phytoplankton microscopic image recognition and counting model based on multi-task learning. End-to-end synchronization achieves accurate identification of algae species and accurate counting of single cells within the colony, thereby overcoming the shortcomings of low efficiency of traditional manual microscopy, large estimation errors of existing area methods, and insufficient generalization ability of customized methods, providing reliable technical support for large-scale water ecological environment monitoring.

[0004] The technical solution of the present invention is as follows:

[0005] The multi-task learning-based phytoplankton microscopic image recognition and cell counting method includes the following steps:

[0006] Step 1: Construct a dataset: Label the phytoplankton species and mark the coordinates of each cell center in the microscopic image. Use the Gaussian kernel function to convert the point annotations into a cell density map.

[0007] Step 2: Feature extraction: ResNet50 is used as the backbone network to extract deep features of microscopic images;

[0008] Step 3, classification branch: input the deep features into the fully connected layer and the Softmax layer, and output the algae species recognition results;

[0009] Step 4, counting branch: perform channel unification and feature extraction on the features output by the backbone network conv3_x, conv4_x, and conv5_x; fuse the multi-scale features from top to bottom through upsampling to generate multi-scale fusion features; input the fusion features into the regression module, output the cell density map and sum it to obtain the cell number;

[0010] Step 5: Model training: Use an alternating training strategy to update parameters, and alternately optimize the classification branch and the counting branch until convergence.

[0011] In the above technical solution, the formula for generating the cell density map is:

[0012] ,

[0013] in, It is the annotation information composed of the center coordinates of each cell in the image. The value of is 1, and the values ​​of the other coordinate points are 0; is of size k and standard deviation Gaussian kernel function.

[0014] In the above technical solution, the multi-scale feature fusion of the counting branch includes: using Convolution unifies the number of feature channels output by conv3_x, conv4_x, and conv5_x modules, using Convolution on compressed features After further feature extraction, the features of three different spatial resolutions are additively fused from top to bottom through upsampling technology.

[0015] In the above technical solution, the regression module is Convolution and Convolution gradually compresses the number of channels of multi-scale fusion features to 1.

[0016] In the above technical solution, the loss function includes:

[0017] Cross entropy loss for the classification branch:

[0018] ,

[0019] in, and are the model predicted category and true category of phytoplankton microscopic image i, and N is the total number of images;

[0020] Mean squared error loss of the counting branch:

[0021] ,

[0022] in, and The true density value and predicted density value at point i in the depth feature map are respectively, is the number of pixels in the depth feature map.

[0023] In the above technical solution, the alternating training strategy is: freezing the counting branch / classification branch parameters in turn, and alternatingly updating the classification branch + backbone network and counting branch + backbone network parameters.

[0024] In the above technical solution, the statistical method for the number of cells is: summing up all pixel values ​​of the predicted cell density map.

[0025] In the above technical solution, the Gaussian kernel parameters are set as: Gaussian kernel size is 25 and standard deviation is 4.

[0026] An electronic device includes a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above method when executing the program.

[0027] A computer-readable storage medium stores a computer program, which implements the above method when executed by a processor.

[0028] Beneficial effects:

[0029] 1. High-precision counting and recognition: Solve problems of cell adhesion and morphological changes through multi-scale feature fusion and density map regression.

[0030] 2. Strong generalization ability: Applicable to 16 types of phytoplankton such as Aphanizomenon and Distella, without the need for customized design.

[0031] 3. Improved efficiency: End-to-end synchronous output of recognition and counting results replaces inefficient manual microscopic inspection.

[0032] 4. Robustness of the method: Compared with the area method, the mean absolute error and mean square error of cell counting are reduced, and the counting accuracy is improved; it still maintains a high accuracy for algae species with overlapping cells and complex population structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a model structure diagram of the present invention.

[0034] Figure 2 1 and 2 are examples of microscopic images, annotated cell density maps, and estimated cell density maps of each category in the examples. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. However, the following embodiments are intended only to explain the present invention, and the scope of protection of the present invention should include the entire contents of the claims. Moreover, through the description of the following embodiments, those skilled in the art can fully implement the entire contents of the claims of the present invention.

[0036] Example 1

[0037] This example uses ResNet50 as the backbone network to extract deep features from microscopic images of algae colony cells. The deep features are then fed into two parallel processing branches, the classification branch and the counting branch, respectively, to achieve accurate identification of algae species and accurate counting of colony cell numbers. The overall network structure of the proposed model is shown in the figure below. Figure 1 As shown, the specific implementation steps are as follows:

[0038] 1. Constructing a phytoplankton microscopic image dataset

[0039] The phytoplankton species were labeled, and the center coordinates of each algal cell in the microscopic image were annotated using tools such as LabelMe. The labeled data of the cell center coordinates were converted into an algal cell density map using the Gaussian kernel function, as shown in formula (1).

[0040] (1)

[0041] in, It is the annotation information composed of the center coordinates of each cell in the image. The value of is 1, and the values ​​of the other coordinate points are 0; is of size k and standard deviation Gaussian kernel function of cell density map The number of algal cells can be obtained by summing them up. In the present invention, the Gaussian kernel size is set to 25 and the standard deviation is set to 4.

[0042] 2. Shared backbone network

[0043] The present invention uses ResNet50 as the backbone network for extracting morphological features from phytoplankton microscopic images. This backbone network consists of several components, conv1, conv2_x, conv3_x, conv4_x, and conv5_x, forming a microscopic image feature extraction module shared by the algae species identification branch and the colony cell counting branch.

[0044] 3. Algae species identification branch

[0045] The algae morphological features extracted by the shared backbone network are input into the fully connected layer and the Softmax layer for algae species identification. The loss function of the algae species identification branch adopts the cross entropy loss, as shown in formula (2).

[0046] (2)

[0047] in, and are the model predicted category and true category of phytoplankton microscopic image i, and N is the total number of images.

[0048] 4. Population Cell Counting Branch

[0049] Use separately Convolution unifies the number of feature channels output by conv3_x, conv4_x, and conv5_x modules (as shown in formula (3)), using Convolution on compressed features After further feature extraction, the features of three different spatial resolutions are additively fused from top to bottom through upsampling technology to obtain multi-scale fusion features of cell morphology, thereby improving the robustness of the model to different morphological features and overlapping cells.

[0050] (3)

[0051] in, The image features extracted by conv1, conv2_x, conv3_x, conv4_x, and conv5_x modules respectively, feature_size is The number of feature channels of the convolution output, in this case .

[0052] Then, the multi-scale fusion features are input into the algae microscopic image cell density regression module, which is Convolution and Convolution gradually compresses the number of channels of the multi-scale fusion feature from feature_size to 1, thereby building a regression model to predict the cell density at each spatial location. The cell density map output by the regression module is accumulated and summed to obtain the number of algae cells in the microscopic image. The loss function of the algae population cell counting branch adopts the mean square error loss , as shown in formula (4).

[0053] (4)

[0054] in, and The true density value and predicted density value at point i in the depth feature map are respectively, is the number of pixels in the depth feature map.

[0055] 5. Model training

[0056] This method discloses a multi-task learning-based phytoplankton microscopic image recognition and cell counting method. Its model architecture consists of three components: a shared backbone network, an algae species recognition branch, and a swarm cell counting branch. During the model training phase, the shared backbone network is initialized using the parameters of a pre-trained ResNet50 model. Subsequently, the learning rate for model training is set to a fixed-step decay strategy, and the model parameters are updated using an alternating training strategy: the two task branches are trained alternately, first updating the parameters of the algae species recognition branch and the shared backbone network, then switching to the swarm cell counting branch and the shared backbone network for parameter updates. This cycle repeats (recognition -> counting -> recognition -> counting...) until the model converges.

[0057] Example 2

[0058] This embodiment discloses a multi-task learning-based method for phytoplankton microscopic image recognition and cell counting, which can simultaneously output phytoplankton species and cell count. This embodiment will illustrate the cell counting performance of the invention from four perspectives: mean absolute error (MAE), mean square error (MSE), counting accuracy per image (Ca_img), and counting accuracy (Ca). The calculation formulas are shown in formulas (5)-(8).

[0059] (5)

[0060] (6)

[0061] (7)

[0062] (8)

[0063] Among them, the following table c is the species category of algae, is the number of images of category c, is the predicted number of cells for image i in category c, is the true number of cells in image i in category c.

[0064] Taking 16 categories of phytoplankton such as Aphanizomenon and Asterix as the objects, the dataset was divided into training set and test set with a ratio of 8:2. The mean absolute error (MAE) of the test set was 1.3510, the mean square error (MSE) was 2.3328, the counting accuracy of each image (Ca_img) was 0.8927, the overall counting accuracy (Ca) was 0.9452, and the recognition accuracy (Ra) was 0.9940. The specific results of each category are shown in Table 1.

[0065] Table 1 Species identification and population cell count results of 16 categories of phytoplankton

[0066]

[0067] The comparison results with the area method counting results (as shown in Table 2) show that this method has obvious advantages in the mean absolute error, mean square error, counting accuracy of each image, and overall counting accuracy of phytoplankton population cell counting, and can simultaneously achieve accurate identification of phytoplankton species and categories.

[0068] Table 2 Comparison results with the area method

[0069]

[0070] Figure 2 Examples of microscopic images, annotated cell density maps, and estimated cell density maps using this method are provided for each category. The results demonstrate the accuracy of cell localization, with the predicted density map's thermal distribution highly concordant with the true cell center. For algae with regular structures, the predicted density map exhibits clear, discrete peaks. For irregular colonies, multi-scale feature fusion effectively captures the fragmented cell distribution. The summed density map results are consistent with the data in Table 1.

[0071] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A phytoplankton microscopic image recognition and cell counting method based on multi-task learning, characterized in that: The following steps are involved: Step 1: Construct a dataset: Label the phytoplankton species and mark the coordinates of each cell center in the microscopic image. Use the Gaussian kernel function to convert the point annotations into a cell density map. Step 2: Feature extraction: ResNet50 is used as the backbone network to extract deep features of microscopic images; Step 3, classification branch: input the deep features into the fully connected layer and the Softmax layer, and output the algae species recognition results; Step 4, counting branch: perform channel unification and feature extraction on the features output by the backbone networks conv3_x, conv4_x, and conv5_x; fuse the multi-scale features from top to bottom through upsampling to generate multi-scale fusion features; The fused features are input into the regression module, the cell density map is output and the sum is calculated to get the cell number; The multi-scale feature fusion of the counting branch includes: Convolution unifies the number of feature channels output by conv3_x, conv4_x, and conv5_x modules, using Convolution on compressed features After further feature extraction, the features of three different spatial resolutions are fused top-down by upsampling technology; The regression module is passed Convolution and Convolution gradually compresses the number of channels of multi-scale fusion features to 1; Step 5: Model training: Use an alternating training strategy to update parameters, and alternately optimize the classification branch and the counting branch until convergence.

2. The method according to claim 1, characterized in that The formula for generating the cell density map is: ， in, It is the annotation information composed of the center coordinates of each cell in the image. The value of is 1, and the values ​​of the other coordinate points are 0; is of size k and standard deviation Gaussian kernel function.

3. The method according to claim 1, characterized in that The loss functions include: Cross entropy loss for the classification branch: ， in, and are the model predicted category and true category of phytoplankton microscopic image i, and N is the total number of images; Mean squared error loss of the counting branch: ， in, and The true density value and predicted density value at point i in the depth feature map are respectively, is the number of pixels in the depth feature map.

4. The method according to claim 1, wherein The alternating training strategy is: freeze the counting branch / classification branch parameters in turn, and alternately update the classification branch + backbone network and counting branch + backbone network parameters.

5. The method according to claim 1, wherein The statistical method for the number of cells is to sum up all pixel values ​​of the predicted cell density map.

6. The method according to claim 1, wherein The Gaussian kernel parameters are set as: Gaussian kernel size is 25 and standard deviation is 4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images