YOLO-based plankton identification method, system and device

Abstract

The invention relates to the technical field of computer deep learning, in particular to a YOLO-based plankton recognition method, system and device, and the method comprises the steps: constructing a sample library of planktonic algae images; constructing a basic detection model based on a YOLO model; according to the planktonic algae image data in the sample library, training the basic detection model to obtain a target detection model; and according to the target detection model, identifying a planktonic algae image. According to the method, the YOLOv8 algorithm backbone network is combined with the residual neural network and the small target detection layer, local and global feature information of the aquatic floating algae image is extracted respectively, the ability of the target detection model to extract aquatic floating algae image data is improved, the problems of small target detection and scale change are solved, the time overhead of the model is reduced, and the detection efficiency is improved. And the target detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer deep learning technology, and in particular to a plankton recognition method, system and device based on YOLO. Background Art

[0002] The automatic identification and counting system of phytoplankton should solve the problem of intelligent identification and automatic counting of aquatic phytoplankton in the collected samples, so as to study the changing characteristics of the community structure and comprehensive evaluation of water quality in the waters where the samples are located. Intelligent identification and automatic counting of phytoplankton are important means to solve this problem. At present, there are mainly two identification methods: morphological-based manual identification of phytoplankton and DNA-based phytoplankton identification. Among them, the manual identification method is a traditional, manual, and expert experience-based method. It classifies and identifies by observing some biological morphological characteristics, such as color, shape, size, texture, organs, etc. This process does not require advanced instruments and equipment, is low-cost, and has a high accuracy rate for some common species. However, this method also has some disadvantages, such as the need for manual identification, relying on manual identification and counting of phytoplankton species under a microscope, subjective errors, low efficiency, and small species coverage. The DNA-based phytoplankton identification method mainly uses the differences in DNA sequences between species for classification and identification. The advantage of this method is that it can accurately identify species and even subspecies, and can also identify unknown species. However, this method also has some disadvantages, such as high equipment cost, high technical requirements, and greater operational difficulty. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent, and proposes a plankton recognition method based on YOLO, comprising:

[0004] Build a sample library of phytoplankton images;

[0005] Build a basic detection model based on the YOLO model;

[0006] According to the phytoplankton image data in the sample library, the basic detection model is trained to obtain a target detection model;

[0007] The phytoplankton image is identified according to the target detection model.

[0008] Preferably, constructing a sample library of phytoplankton images includes:

[0009] The images of phytoplankton in the same field of view at different focal lengths are collected by an imaging system;

[0010] Based on the multi-focus image fusion algorithm, a plurality of phytoplankton images collected are subjected to image fusion processing to obtain a phytoplankton fusion image;

[0011] All fusion images of phytoplankton are annotated and a sample library is constructed.

[0012] Preferably, the method for acquiring the phytoplankton image further comprises: collecting the phytoplankton image under a microscope by means of online image search.

[0013] Preferably, the basic detection model includes: a first feature extraction module, a fusion module, a second feature extraction module and a YOLOV8 backbone network connected in sequence.

[0014] Preferably, the first feature extraction module includes: five Resblocks connected in sequence, a layer of spatial attention module and a layer of channel attention module; wherein the Resblock includes two layers of Conv2d and LeakyReLU.

[0015] Preferably, the fusion module is a UFA module.

[0016] Preferably, an attention mechanism module and a small target detection module are further provided between the second feature extraction module and the YOLOV8 backbone network; the attention mechanism module includes: an average pooling layer, two linear layers, a ReLU activation function and a Sigmoid function.

[0017] The present disclosure also provides a plankton recognition system based on YOLO, including:

[0018] The first building module is used to build a sample library of phytoplankton images;

[0019] The second building module is used to build a basic detection model based on the YOLO model;

[0020] A model training module, used to train the basic detection model according to the phytoplankton image data in the sample library to obtain a target detection model;

[0021] An image recognition module is used to recognize phytoplankton images according to the target detection model.

[0022] The present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program or instruction, and when the computer program or instruction is executed by the processor, it is at least used to implement the above method.

[0023] The present disclosure also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, it is at least used to implement the above method.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention uses the YOLOv8 algorithm backbone network, combines it with the residual neural network and the small target detection layer, respectively extracts the local and global feature information of the aquatic phytoplankton image, improves the ability of the target detection model to extract aquatic phytoplankton image data, solves the problems of small target detection and scale change, reduces the time overhead of the model, improves the target detection accuracy, and combines the research on deep learning target detection with plankton detection in a specific ecological field, which provides a feasible and effective method for actual plankton detection, and is of great significance.

[0025] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings. The technical scheme of the present invention is further described below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 A schematic diagram of a plankton identification method based on YOLO is provided in an embodiment;

[0028] Figure 2 A flow chart for establishing a phytoplankton image sample library given in the embodiment;

[0029] Figure 3 A structural diagram of a basic detection model provided in an embodiment;

[0030] Figure 4 A schematic diagram of a plankton recognition system based on YOLO is provided in the embodiment;

[0031] Figure 5 A schematic diagram of an electronic device provided for an embodiment;

[0032] Figure 6 A schematic diagram of a computer-readable storage medium is provided for an embodiment. DETAILED DESCRIPTION

[0033] The present invention is described below in conjunction with the accompanying drawings. The preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0034] Figure 1 The plankton identification method based on YOLO provided by the present invention includes:

[0035] Build a sample library of phytoplankton images;

[0036] Build a basic detection model based on the YOLO model;

[0037] According to the phytoplankton image data in the sample library, the basic detection model is trained to obtain a target detection model;

[0038] The phytoplankton image is identified according to the target detection model.

[0039] According to some embodiments of the present application, constructing a sample library of phytoplankton images includes:

[0040] The images of phytoplankton in the same field of view at different focal lengths are collected by an imaging system;

[0041] Based on the multi-focus image fusion algorithm, a plurality of phytoplankton images collected are subjected to image fusion processing to obtain a phytoplankton fusion image;

[0042] All fusion images of phytoplankton are annotated and a sample library is constructed.

[0043] According to some embodiments of the present application, the method for acquiring phytoplankton images further includes: collecting phytoplankton images under a microscope by means of online image search.

[0044] According to some embodiments of the present application, the basic detection model includes: a first feature extraction module, a fusion module, a second feature extraction module and a YOLOV8 backbone network connected in sequence; wherein the fusion module is a UFA module, the first feature extraction module includes: five Resblocks connected in sequence, a layer of spatial attention module and a layer of channel attention module; Resblock includes two layers of Conv2d and LeakyReLU;

[0045] According to some embodiments of the present application, an attention mechanism module and a small target detection module (STL module) are also provided between the second feature extraction module and the YOLOV8 backbone network; the attention mechanism module includes: an average pooling layer, two linear layers, a ReLU activation function and a Sigmoid function. The STL module is a small target detection layer. The STL module slices the picture of the phytoplankton before it enters the backbone network. The specific operation is to get a value for every other pixel in a picture, which is similar to neighboring downsampling. In this way, four times the number of pictures are obtained without information loss, and the W and H information are converted into channel information. The input channel is expanded by 4 times, that is, the spliced ​​picture becomes 12 channels relative to the original RGB three-channel mode. Finally, the new picture is subjected to a convolution operation, and finally a double downsampling feature map without information loss is obtained.

[0046] According to an embodiment of the present application, a plankton recognition method based on YOLO includes:

[0047] (1) Establish an imaging system to collect phytoplankton image data

[0048] An imaging system is used to take images of water samples under a microscope to obtain image data of phytoplankton, including images of different types, sizes, angles, and background conditions, so as to realize offline collection of data required for model training. At the same time, online image search and other means are used to collect phytoplankton image data under a microscope and establish a morphological feature dataset of phytoplankton images; unlike most public datasets, a microscope is used to take pictures of algae in the same field of view at multiple focal lengths during the collection of phytoplankton, so multiple pictures of algae with the same content will be produced. The characteristic of multi-focus shooting is that the clear parts of the same picture taken at different focal lengths are different, and since the technical purpose of this application is to perform target detection and determine the types and quantities of phytoplankton in water samples, this implementation scheme learns and draws on the deep learning related ideas of feature extraction and feature fusion in the multi-focus image fusion algorithm, selects the UFA-Fusion algorithm for improvement, and extracts and fuses features of images at different focal lengths in the same field of view without generating traditional decision diagrams and without the need for post-processing. The final fused image achieves a clear expression of most objects in the image, which is helpful for subsequent model training and target detection.

[0049] Since existing image fusion algorithms are mostly trained using foreground, background and panoramic images, and are only divided into foreground focus and background focus when testing image fusion, and the imaging system captures phytoplankton images from 3 / 5 focal lengths, the model needs to be improved and processed, and changed to a 5-image input network structure to adapt to actual conditions.

[0050] According to some embodiments of the present application, in order to better perform image fusion, DUTS-TE is used to perform Gaussian blur and binarization operations on the data set, and 5 images at different focal lengths are generated for each image. Finally, a multi-focus image data set is established to obtain pre-trained weights, which facilitates image fusion of phytoplankton images and extracts more features.

[0051] (2) Establishing a sample library of phytoplankton images

[0052] The fused phytoplankton image data is preprocessed, including image denoising, image enhancement, image segmentation and other operations, to improve the accuracy and speed of subsequent feature extraction and target detection. The sample library construction flow chart is shown in the figure. Figure 2 As shown: First, an imaging system is used to collect multiple images, and the images are checked for duplication, distributed, reviewed, and annotated. The annotated images are processed to make them clearer and more suitable for model training. Then, a phytoplankton image dataset is formed, and a phytoplankton image sample library is constructed.

[0053] (3) Constructing a phytoplankton image feature target detection model

[0054] The improved YOLOv8 model is used to extract morphological features such as shape, color, and texture from the preprocessed phytoplankton images to obtain the phytoplankton image feature target detection model, and model evaluation and optimization are performed, including parameter adjustment, network structure optimization, data enhancement and other operations, to improve the accuracy and robustness of the model.

[0055] refer to Figure 3 The feature extraction and feature fusion of the improved model include 5 Resblocks (two layers of Conv2d and LeakyReLU), 1 layer of spatial attention module (SpatialAttention) and 1 layer of channel attention module (ChannelAttention). The fusion module uses the UFA module (Conv2d and LeakyReLU) for feature fusion. In the process of collecting phytoplankton images, due to different focal lengths, a large part of the images have the phenomenon of local overexposure of the edge of the object, which affects the subsequent feature extraction for target detection. The feature extraction and feature fusion modules after the improvement of image fusion are more sensitive to the edge of the object, and can obtain clearer object edges than the image before fusion, which is helpful for subsequent target detection operations.

[0056] The SE attention mechanism and small target detection layer are added to the Yolov8 backbone network. In order to make the target detection algorithm pay more attention to the morphological characteristics of phytoplankton, the SE attention mechanism is used. The average pooling layer is used for average pooling, and there are 2 linear layers, 1 ReLU activation function, and 1 Sigmoid function. The attention mechanism enables the deep learning network to focus more on key information that helps target detection. In the actual phytoplankton dataset, the proportion of algae is small, and there is a phenomenon of aggregation and clustering. At the same time, due to the underwater image shooting, there are bubbles in the image, which makes it difficult to accurately detect some small phytoplankton in the target detection task. In order to solve this problem, a small target detection layer is added to the network structure so that the network can better detect small targets. The small target detection layer usually includes multiple small-scale detection frames. By detecting at different scales, the accuracy of small target detection can be improved.

[0057] Use RtdertDecoder to replace the original detection head of yolov8. Use a hybrid encoder to decouple the intra-scale interaction and cross-scale fusion method, which can process multi-scale features more effectively compared with the traditional NMS algorithm.

[0058] (4) Establish an online detection system for intelligent identification of aquatic phytoplankton

[0059] Through the phytoplankton image feature target detection model obtained in the above process, an intelligent online detection system for aquatic phytoplankton is established to perform target detection on phytoplankton images, obtain the types and numbers of phytoplankton contained in the sample, and thus calculate indicators such as species richness and uniformity index and output a detection report.

[0060] Figure 4 The YOLO-based plankton recognition system provided for this application includes:

[0061] The first building module is used to build a sample library of phytoplankton images;

[0062] The second building module is used to build a basic detection model based on the YOLO model;

[0063] A model training module, used to train the basic detection model according to the phytoplankton image data in the sample library to obtain a target detection model;

[0064] An image recognition module is used to recognize phytoplankton images according to the target detection model.

[0065] like Figure 5As shown, the present application provides an electronic device 1000, which includes a memory 1002 and a processor 1001. The memory 1002 stores a computer program or instruction. When the computer program or instruction is executed by the processor 1001, it is at least used to implement the above method. Figure 6 As shown, the present application provides a computer-readable storage medium 1100, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, it is at least used to implement the above method.

[0066] It is obvious that a person skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A plankton identification method based on YOLO, characterized in that: include: Build a sample library of phytoplankton images; Build a basic detection model based on the YOLO model; According to the phytoplankton image data in the sample library, the basic detection model is trained to obtain a target detection model; The phytoplankton image is identified according to the target detection model.

2. The plankton identification method based on YOLO as claimed in claim 1, characterized in that: Build a sample library of phytoplankton images, including: The images of phytoplankton in the same field of view at different focal lengths are collected by an imaging system; Based on the multi-focus image fusion algorithm, a plurality of phytoplankton images collected are subjected to image fusion processing to obtain a phytoplankton fusion image; All fusion images of phytoplankton are annotated and a sample library is constructed.

3. The plankton identification method based on YOLO as claimed in claim 2, characterized in that: The method for acquiring the phytoplankton image also includes: collecting the phytoplankton image under a microscope by online image search.

4. The plankton identification method based on YOLO as claimed in claim 1, characterized in that: The basic detection model includes: a first feature extraction module, a fusion module, a second feature extraction module and a YOLOV8 backbone network connected in sequence.

5. The plankton identification method based on YOLO as claimed in claim 4, characterized in that: The first feature extraction module includes: five Resblocks connected in sequence, a layer of spatial attention module and a layer of channel attention module; wherein, The Resblock includes two layers of Conv2d and LeakyReLU.

6. The plankton identification method based on YOLO as claimed in claim 4, characterized in that: The fusion module is a UFA module.

7. The plankton identification method based on YOLO as claimed in claim 4, characterized in that: An attention mechanism module and a small target detection module are also provided between the second feature extraction module and the YOLOV8 backbone network; the attention mechanism module includes: an average pooling layer, two linear layers, a ReLU activation function and a Sigmoid function.

8. A plankton recognition system based on YOLO, characterized in that: include: The first building module is used to build a sample library of phytoplankton images; The second building module is used to build a basic detection model based on the YOLO model; A model training module, used to train the basic detection model according to the phytoplankton image data in the sample library to obtain a target detection model; An image recognition module is used to recognize phytoplankton images according to the target detection model.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores a computer program or instruction, which, when executed by the processor, is used to implement at least the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction, and when the computer program or instruction is executed by the processor, it is used to implement at least the method according to any one of claims 1 to 7.

Images