A method for detecting phytoplankton microorganisms based on deep learning

Abstract

A method for detecting phytoplankton microorganisms based on deep learning includes the following steps: Step 1: preprocessing the original training data, organizing the data set into unlabeled samples and labeled samples, selecting diatoms, green algae and cyanobacteria as target categories in the labeled samples, and the format meets the COCO format requirements; Step 2: establishing a hybrid neural network structure for fast inference of image classification, combining attention bias and deformable convolution, using the backbone network to extract features, and combining a Deformable DETR detection model with better small target and density effects; Step 3: using the labeled data set as the input of the network; Step 4: training the network: first pre-training with a large number of unlabeled samples; using labeled data sets from diatoms, green algae and cyanobacteria to complete the downstream task of algae target detection. The present invention realizes fast and high-precision phytoplankton target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision. The present invention discloses a hybrid neural network for fast inference of image classification to achieve fast and high-precision detection of phytoplankton targets. The algae microorganism characteristics are learned by using a small number of labeled samples, and the algae microorganisms have diverse morphologies, a wide variety of species, and dense and tiny characteristics. At the same time, a large number of unlabeled samples are used to quickly and effectively apply deep learning methods to algae detection tasks. Background Art

[0002] Traditional methods for detecting phytoplankton microorganisms often require manual intervention and analysis, which is time-consuming, labor-intensive, and prone to subjective errors. It is a difficult, expensive, and time-consuming task. There are many methods for algae detection, including traditional methods such as microscopy and chemical analysis, as well as more advanced technologies such as remote sensing and machine learning. In recent years, deep learning has been widely used in machine vision directions such as image classification and object detection and has made breakthrough achievements. Using deep learning technology, it is possible to achieve automated detection and classification of phytoplankton, reduce the manual burden, and improve the accuracy and efficiency of detection.

[0003] In terms of phytoplankton target detection, algae detection is a challenging task. There are usually three more important and non-ignorable problems. First, algae microorganisms are highly variable in appearance, with various shapes, sizes, and colors, which increases the difficulty of the model to accurately detect all types of algae. In addition, algae are often gathered in dense clusters or mats, which makes it difficult to distinguish individual microorganisms from the background, resulting in possible false positives or false negatives during the detection process. Another problem is that there is limited annotated data available for training deep learning models. There are usually few labeled images of algae microorganisms, especially for very rare species, and there are many species. The model cannot recognize when encountering new algae microorganisms, and may not achieve the effect of water quality monitoring. This makes it difficult to develop accurate models for algae detection and also limits the generalization ability of models trained on small data sets.

[0004] LeVit is a hybrid neural network for fast inference of image classification. It combines the advantages of convolutional neural networks and Transformer models and accelerates model training by gradually reducing the resolution of activation maps. At the same time, LeVit introduces attention bias, a new method that effectively integrates position information into visual Transformers. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art and to achieve a fast, accurate and efficient method for automatic detection and identification of algae microorganisms, the present invention adopts a new backbone network and combines the Deformable DETR method suitable for small targets and dense scenes to achieve fast and high-precision phytoplankton target detection.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A method for detecting phytoplankton microorganisms based on deep learning, the method comprising the following steps:

[0008] Step 1: Preprocess the original training data and organize the data set into a large number of unlabeled samples and labeled samples. Among the labeled samples, diatoms, green algae and cyanobacteria are selected as target categories. The format of the data set meets the requirements of the coco format;

[0009] Step 2: Establish a hybrid neural network structure for fast inference of image classification, and use the backbone network to extract features;

[0010] Step 3: Use the labeled dataset obtained in step 1 as the input of the network in step 2;

[0011] Step 4: Network training: In the first stage, a large number of unlabeled samples were used for pre-training; followed by the second stage of fine-tuning, using labeled datasets from Bacillariophyta, Chlorophyta, and Cyanobacteria to complete the downstream task of algae target detection.

[0012] Furthermore, in step 1, the data set division steps are as follows:

[0013] 1.1. Take unlabeled samples and labeled samples from the original training set in turn;

[0014] 1.2. Among the labeled samples, we selected the data of diatoms, green algae and cyanobacteria, and cleaned up the duplicate and harmful data;

[0015] 1.3. Convert the format of the screened labeled samples from labelme to coco format dataset for target detection;

[0016] 1.4. Divide the training data set into a 3:7 test and training data set according to the standard training data set obtained in step 3 as the input of the network.

[0017] Furthermore, in step 2, the network construction process is as follows:

[0018] The structure of patch embedding: It consists of 5 layers in total. The first 4 layers correspond to 3*3 convolution layers. The 5th layer consists of a maximum pooling layer + h-swish activation layer. The convolution kernel size of all convolution layers is 3×3, the convolution step size is 2, and the padding is 1. The maximum pooling layer also has a convolution kernel size of 3×3, a convolution step size of 2, and a padding of 1.

[0019] Stage structure: In each stage, a DCNv3 operator is stacked, and after the LN operation, the residual at the time of input is added, and then a self-attention module and an MLP module are passed, and the residual at the time of input of the self-attention module is added;

[0020] Activation layer function expression:

[0021] ReLu activation function:

[0022]

[0023] Where x represents input and f(x) represents output;

[0024] h-swish activation function:

[0025]

[0026] Where x represents the input and f(x) represents the output.

[0027] This invention proposes a new model architecture, which is beneficial to the direct global relationship modeling ability of Transformer, realizes input adaptation and long-range dependency, and combines the characteristics of fast training and faster convergence of CNN. That is, it adopts the combination of DCN and attention mechanism of LeVit, CNN Backbone, and DeformableDETR with better effect on small targets and density, to realize fast and high-precision phytoplankton target detection method.

[0028] The beneficial effect of the present invention is: using a large amount of unlabeled data and labeled data to learn useful features, so as to achieve fast and high-precision phytoplankton target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the small target algae picture of the present invention;

[0030] Figure 2 This is the overall principle framework diagram of the small target detection network;

[0031] Figure 3 This is a comparison chart of the final original image and detected image of the entire detection model, where the left picture is the original image and the right picture is the detected image. DETAILED DESCRIPTION

[0032] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0033] Reference Figure 1 to Figure 3 , a method for detecting phytoplankton microorganisms based on deep learning, the method comprising the following steps:

[0034] Step 1: Preprocess the original training data and organize the data set into a large number of unlabeled samples and labeled samples. Diatoms, green algae and cyanobacteria are selected as target categories in the labeled samples. The format of the data set meets the requirements of the coco format.

[0035] In step 1, the data set is preprocessed and the data set is divided as follows:

[0036] 1.1. Take unlabeled samples and labeled samples from the original training set in turn;

[0037] 1.2. Among the labeled samples, we selected the data of diatoms, green algae and cyanobacteria, and cleaned up the duplicate and harmful data;

[0038] 1.3. Convert the format of the screened labeled samples from labelme to coco format dataset for target detection;

[0039] 1.4. Divide the training data set into a 3:7 test and training data set according to the standard training data set obtained in step 3 as the input of the network;

[0040] Step 2: Establish a hybrid neural network structure for fast inference of image classification, and use the backbone network to extract features;

[0041] In step 2, the network construction process is as follows:

[0042] The structure of patch embedding: It consists of 5 layers in total, of which the first 4 layers correspond to 3*3 convolution layers. The 5th layer consists of a maximum pooling layer + h-swish activation layer, where the convolution kernel size of all convolution layers is 3×3, the convolution step size is 2, and the padding is 1. The maximum pooling layer also has a convolution kernel size of 3×3, a convolution step size of 2, and a padding of 1.

[0043] Stage structure: In each stage, a DCNv3 operator is stacked, and after the LN operation, the residual at the time of input is added. Then, a self-attention module and an MLP module are passed, and the residual at the time of input of the self-attention module is added;

[0044] Activation layer function expression:

[0045] ReLu activation function:

[0046]

[0047] Where x represents the input and f(x) represents the output.

[0048] h-swish activation function:

[0049]

[0050] Where x represents the input and f(x) represents the output.

[0051] Step 3: Use the labeled dataset obtained in step 1 as the input of the network in step 2;

[0052] Step 4: Network training: In the first stage, a large number of unlabeled samples were used for pre-training. Then, the second stage of fine-tuning was carried out, using labeled datasets from Bacillariophyta, Chlorophyta, and Cyanobacteria to complete the downstream task of algae target detection.

[0053] The method for detecting phytoplankton microorganisms based on deep learning in this embodiment comprises the following steps:

[0054] Step 1: Preprocess the original training data and organize the data set into a large number of unlabeled samples and labeled samples. Diatoms, green algae and cyanobacteria are selected as target categories in the labeled samples. The cleaned labeled samples are formatted and converted from labelme to coco format dataset for target detection.

[0055] Step 2: Establish a hybrid neural network structure B for fast inference of image classification. After Patch embedding, the overall network depth is 3 layers, the number of stacked stages in each layer is 1, 1, 2, the number of self-attention heads used in each stage is 3, 5, 6, the dimension of Key is 16, and multi-scale output is used. Finally, 3 scales are output as neck input, which are 128, 256, and 384 respectively. Deformable Detr network structure H;

[0056] Step 3: Input the training data set divided according to 3:7 into backbone network B, and then use the output of backbone network B as input into network structure H.

[0057] Step 4: Train for 50 epochs with a batch size of 8, using the Adam optimizer, hyperparameters of 2e-4, and momentum of 0.9.

[0058] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A method for detecting phytoplankton microorganisms based on deep learning, characterized in that: The method comprises the following steps: Step 1: Preprocess the original training data and organize the data set into a large number of unlabeled samples and labeled samples. Among the labeled samples, diatoms, green algae and cyanobacteria are selected as target categories. The format of the data set meets the COCO format requirements. Step 2: Establish a hybrid neural network structure for fast inference of image classification, and use this network to extract features; Step 3: Use the labeled dataset obtained in step 1 as the input of the network in step 2; Step 4: Network training: In the first stage, a large number of unlabeled samples were used for pre-training; then the second stage of fine-tuning was carried out, using labeled datasets from Bacillariophyta, Chlorophyta, and Cyanobacteria to complete the downstream task of algae target detection; In step 2, the network construction process is as follows: The structure of patch embedding: It consists of 5 layers in total. The first 4 layers correspond to 3*3 convolution layers. The 5th layer consists of a maximum pooling layer + h-swish activation layer. The convolution kernel size of all convolution layers is 3×3, the convolution step size is 2, and the padding is 1. The maximum pooling layer also has a convolution kernel size of 3×3, a convolution step size of 2, and a padding of 1. Stage structure: In each stage, a DCNv3 operator is stacked, and after the LN operation, the residual at the time of input is added, and then a self-attention module and an MLP module are passed, and the residual at the time of input of the self-attention module is added; Activation layer function expression: ReLu activation function: Where x represents input and f(x) represents output; h-swish activation function: ReLU6=min(ReLU,6) Where x represents the input and f(x) represents the output.

2. A method for detecting phytoplankton microorganisms based on deep learning as claimed in claim 1, characterized in that: In step 1, the steps of dividing the data set are as follows: 1.

1. Take unlabeled samples and labeled samples from the original training set in turn; 1.

2. Among the labeled samples, we selected the data of diatoms, green algae and cyanobacteria, and cleaned up the duplicate and harmful data; 1.

3. Convert the format of the screened labeled samples from labelme to COCO format dataset for target detection; 1.

4. Divide the training data set into a test data set with a ratio of 3:7 according to the standard training data set obtained in 1.3, and use it as the input of the network.

Images