Deep learning-based zooplankton classification and counting method and system
Through deep learning-based methods, image data is used to train convolutional neural networks and YOLOv7 models, the fast and accurate classification and counting of zooplankton is achieved, and the problems of inefficient or high cost in the monitoring methods in the existing technology are solved, and graphical output is provided for user analysis.
Patent Information
- Application Number
- CN202510057073.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-05-30
AI Technical Summary
In the prior art, the identification and classification of zooplankton relies on artificial microscopy, which is time-consuming and labor-intensive, and the monitoring methods are inefficient or expensive, and lack long-term high-frequency and low-cost monitoring methods.
Using a deep learning-based method, image data is obtained by taking pictures of zooplankton samples at different depths and locations, preprocessing and dividing them into training sets, verification sets and test sets. The convolutional neural network model is trained using the PyTorch framework, and combined with the YOLOv7 object detection network for classification and counting, and the results are integrated and visualized.
It realizes fast and accurate classification and counting of zooplankton, reduces hardware requirements, provides graphical output, facilitates user understanding and analysis, and solves the problems of inefficient or high cost in the existing technology.
Smart Images
Figure CN120070950A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image feature recognition, and particularly to a method and system for classifying and counting zooplankton based on deep learning. Background Art
[0002] Water eutrophication is one of the current major ecological environment problems, threatening global water security. The number of algal cells per unit volume of water is an important indicator of the degree of water eutrophication. The species and quantity of zooplankton are two important indicators for predicting the outbreak of water blooms or red tides, and their monitoring is very important for the prevention and control of eutrophication. In the past, the identification and classification of zooplankton were generally completed by very time-consuming and laborious manual microscopic examination after taking water samples. The existing monitoring means are still relatively inefficient or costly, and there is an urgent need for a monitoring method that can be long-term, high-frequency, and low-cost. Summary of the Invention
[0003] The main object of the present invention is to provide a method, system, device, and storage medium for classifying and counting zooplankton based on deep learning, aiming to solve the technical problems in the prior art.
[0004] To achieve the above object, the present invention provides a method for classifying and counting zooplankton based on deep learning, the method comprising the following steps: S1, taking pictures of the zooplankton samples at different depths and positions to obtain image data of the zooplankton; S2, preprocessing the image data to obtain preprocessed image data, and dividing the data into training set data, validation set data, and test set data according to a preset ratio; S3, based on the deep learning framework PyTorch, inputting the training set data and validation set data to train a convolutional neural network model and extract feature vectors; S4, based on the feature vectors, using the softmax function to output classification probabilities to obtain the classification results of the zooplankton; and based on the predetermined object detection network YOLOv7, performing positioning and counting on the preprocessed image data to obtain the counting results of each type of zooplankton; S5, integrating the classification results and the counting results, and visually outputting the integrated results.
[0005] Preferably, a high-definition CMOS camera and a trinocular stereo microscope are used to take pictures and collect the zooplankton samples at different depths and positions to obtain the image data of the zooplankton.
[0006] Preferably, the preprocessing includes: respectively performing annotation, size adjustment, grayscale conversion, contrast enhancement, and image denoising on the image data.
[0007] Preferably, a picture preset annotation tool LabelImg or Vott is used for data annotation to obtain the annotated image data;
[0008] Perform size adjustment on the marked image data, where the size adjustment includes setting the size of the cropped sub-image to have a length and width within the range of 300 - 400 pixels.
[0009] Perform grayscale conversion on the target sub-image using the weighted average method to obtain the grayscale sub-image; perform contrast enhancement on the grayscale sub-image using the histogram equalization method to obtain the contrast-enhanced sub-image; perform image denoising on the contrast-enhanced sub-image using the median filtering method to obtain the preprocessed image data.
[0010] Preferably, the convolutional neural network model includes a batch normalization layer, multiple convolutional layers, multiple average pooling layers, a Dropout layer, a flatten layer, a fully connected layer, and an output layer. The first layer is a convolutional layer, the second layer is a batch normalization layer, followed by a convolutional layer and a batch normalization layer, then followed by an average pooling layer and a Dropout layer in sequence. The above pattern is repeated multiple times, and finally a flatten layer is added, and a batch normalization layer, a fully connected layer, and an output layer are added after the flatten layer. The convolutional layer contains 64 convolutional kernels of size 5*5, and the ReLU activation function is used.
[0011] Preferably, the improved object detection network YOLOv7 includes four main components: an image input, a backbone network, a neck network, and a head network. The backbone network integrates BConv, ELAN, and MP-1 convolutional modules. The neck network contains ELAN-W and MP-2 modules, and replaces the SPP module with the SPPCSPC module to adapt to different input sizes. The head network uses the RepConv module.
[0012] Preferably, the attention mechanism SimAm is introduced in the neck network, and the WIoU loss function is used to achieve fast convergence.
[0013] In addition, to achieve the above object, the present application also proposes a zooplankton classification and counting system based on deep learning. The system includes: an image data acquisition module for taking pictures of the zooplankton samples at different depths and positions to obtain the image data of the zooplankton.
[0014] An image preprocessing module preprocesses the image data to obtain preprocessed image data, and divides the preprocessed image data into training set data, validation set data, and test set data according to a preset ratio; a feature vector extraction module, based on the deep learning framework PyTorch, inputs the training set data and the validation set data to train a convolutional neural network (CNN), and extracts feature vectors; a classification module, based on the feature vectors, uses the softmax function to output classification probabilities to obtain the classification results of zooplankton; a counting module, based on the predetermined object detection network YOLOv7, locates and counts the preprocessed image data to obtain the counting results of each type of zooplankton; an output module integrates the classification results and the counting results, and visually outputs the integrated results.
[0015] In addition, to achieve the above object, the present application also proposes a computer device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the above-mentioned deep learning-based zooplankton classification and counting method.
[0016] In addition, to achieve the above object, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the above-mentioned deep learning-based zooplankton classification and counting method.
[0017] The present invention takes pictures of the zooplankton samples at different depths and positions to obtain the image data of the zooplankton; preprocesses the image data to obtain preprocessed image data, and divides the data into training set data, validation set data, and test set data according to a preset ratio; inputs the training set data and the validation set data to train an improved convolutional neural network model, and extracts feature vectors; uses the softmax function to output classification probabilities to obtain the classification results of zooplankton; based on the improved object detection network YOLOv7, locates and counts the preprocessed image data to obtain the counting results of each type of zooplankton; integrates the classification results and the counting results, and then visually outputs them. The deep learning model is optimized. Especially in the feature extraction stage, more advanced network structures, such as attention mechanisms and adaptive pooling layers, are introduced, making the model have better robustness and adaptability under different lighting and perspective conditions. In addition, data augmentation techniques are introduced during the model training process to reduce overfitting and bias of the model. At the same time, by introducing interpretability tools, such as feature visualization and attention maps, the interpretability of the model is improved. Description of the Drawings
[0018] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0020] Figure 1 It is a schematic structural diagram of a device in the hardware operating environment related to the embodiment solution of the present invention;
[0021] Figure 2 It is a schematic flowchart of an embodiment of a method for classifying and counting zooplankton based on deep learning according to the present invention;
[0022] Figure 3 It is an image of zooplankton in an embodiment of a method for classifying and counting zooplankton based on deep learning according to the present invention;
[0023] Figure 4 It is a schematic flowchart of a convolutional neural network in an embodiment of a method for classifying and counting zooplankton based on deep learning according to the present invention;
[0024] Figure 5 It is a block diagram of an embodiment of a system for classifying and counting zooplankton based on deep learning according to the present invention.
[0025] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed Embodiments
[0026] It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the present invention.
[0027] Figure 1 It exemplifies a schematic structural diagram of an electronic device, as Figure 1 shown. The electronic device may include: a processor 901, a communication interface 902, a memory 903, and a communication bus 904. Among them, the processor, the communication interface, and the memory communicate with each other through the communication bus. The processor can call the logical instructions in the memory to execute a method for classifying and counting zooplankton based on deep learning.
[0028] Those skilled in the art can understand, Figure 1The structure shown does not constitute a limitation on the computer device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0029] An embodiment of the present invention provides a method for classifying and counting zooplankton based on deep learning. Refer to Figure 3 , Figure 3 which is a schematic flowchart of an embodiment of the method for classifying and counting zooplankton based on deep learning of the present invention.
[0030] In this embodiment, the method for classifying and counting zooplankton based on deep learning includes the following steps:
[0031] S1: Take pictures of the zooplankton samples at different depths and positions to obtain the image data of the zooplankton.
[0032] It should be noted that a high-definition CMOS camera and a trinocular stereomicroscope are used to take pictures of the zooplankton samples at different depths and positions for acquisition to obtain the image data of the zooplankton.
[0033] Using a high-definition CMOS camera: Place the zooplankton sample in a transparent container to ensure that the sample is clearly visible. If necessary, Lugol's solution can be used for staining to improve visibility. According to needs, adjust the position and angle of the camera to simulate observations at different depths. For example, the height of the camera can be changed or the zoom function can be used to simulate the deep-sea environment. Start the camera and perform continuous shooting or single-shot shooting to ensure that there is sufficient image data for each angle and depth. To ensure image quality, multiple shots may be required and the best images selected. Recording and sorting: Make detailed records for each picture, including the depth, position, and other relevant parameters of the shot for subsequent research.
[0034] The main components of the trinocular stereomicroscopic imaging system include a light source controller, a light source module, a flow injector, a glass slide, a tray, a 10x long working distance objective lens, a servo drive, a motion controller, a high-precision micro-displacement Z-axis focusing platform, and a PC host computer (a computer for sending control instructions). Among them, the high-precision micro-displacement Z-axis focusing platform is the most core automated part of the device, capable of achieving micro-displacements at the μm level. In practical applications, the sampling frequency, shooting thickness, moving step size, shutter speed, and signal-to-noise ratio can be set through the PC side. When using this device for zooplankton monitoring, first place the sample on the glass slide, then turn on the light source, and finally set the total distance of the camera movement and the step size of each single movement. The camera obtains multi-layer images at different depths on the glass slide as the high-precision moving platform moves up and down. The images taken by the camera will be automatically stored in the computer connected to the device for subsequent image processing and analysis. For specific images of zooplankton, refer to Figure 2 .
[0035] Step S2: Preprocess the image data to obtain preprocessed image data, and divide the data into training set data, validation set data, and test set data according to a preset ratio.
[0036] It should be noted that when dividing the data into training set data, validation set data, and test set data according to a preset ratio, for a small-scale sample set, the division ratio used is 7:2:1 for the training set, validation set, and test set. For example, if there are a total of 1000 samples, then the training set is divided into 700 samples, the validation set is 200 samples, and the test set is 100 samples.
[0037] It should be noted that the preprocessing includes: respectively performing annotation, size adjustment, grayscale conversion, contrast enhancement, and image denoising on the image data.
[0038] It can be understood that the data is annotated using the picture preset annotation tools LabelImg or Vott to obtain the annotated image data; during the implementation of this system, the picture data needs to be calibrated, and the calibration tool uses an open-source tool. Description of the LabelImg tool: Rectangular annotation, supporting the calibration of the detection box of each line of characters in the picture (calibrating each column for vertical text); Description of the Vott tool: Supporting picture annotations such as rectangles and polygons, supporting video annotation, convenient shortcut keys and a beautiful interface, and also supporting the export of multiple label formats.
[0039] It can be understood that to improve the image processing speed and not lose too much image information due to cropping, in this study, the size of the cropped target sub-image is set to have a length and width within the range of 300 - 400 pixel points;
[0040] It can be understood that the weighted average method is used to perform grayscale conversion on the target sub-image to obtain the grayscale-converted sub-image;
[0041] Specifically, the zooplankton images collected by the device are in RGB mode. The pixel values are divided into three parameters: R (Red), G (Green), and B (Blue), with the minimum component value being 0 and the maximum being 255. The grayscale value is the intensity information of each pixel in the grayscale image, which can be used to describe the brightness and purity of the image. The grayscale range is from 0 to 255. When the grayscale value reaches 255, it represents the brightest (pure white) image; when the grayscale value drops to zero, it represents the darkest (pure black) image. The RGB mode cannot fully capture the morphological features of the image, so other color modes need to be used to process these three color components. The advantages of image grayscale conversion are as follows: it can reduce the memory occupancy in the RGB mode and increase the running speed; it can improve visual contrast and highlight the target area. To speed up the calculation, the RGB mode of the original image is grayscale-converted. Through the weighted average method for grayscale conversion, that is, according to actual needs, three different weight values are set to perform weighted averaging on the three parameters so that the brightness and chromaticity information of the image can be better represented.
[0042] It can be understood that the contrast of the grayscale sub-image is enhanced by using the histogram equalization method to obtain the sub-image with enhanced contrast; specifically, the stretching of the histogram can achieve the imaging effect of increasing the contrast. The operation method is to stretch the peak of the pixel brightness distribution of the histogram while keeping its original shape unchanged, making better use of the display width and increasing the display contrast effect. Histogram stretching operates on all pixel points on the image simultaneously and considers the entire image. Therefore, better results can be obtained.
[0043] It can be understood that the median filtering method is used to denoise the sub-image with enhanced contrast to obtain the preprocessed image data. The median filtering method is a non-linear smoothing method that realizes the smoothing of the image by setting the grayscale value of each image point to the median value of each image point within a certain range. Median filtering is a digital image processing technology that eliminates the noise points in the digital sequence by sorting the pixel values, making them close to the true values, thereby improving the quality of the image. The steps of the algorithm are as follows: Step1: Set the size of the sliding window; Step2: Move the window center to a certain pixel point on the image; Step3: Arrange the pixel points within the window according to the grayscale value; Step4: Take the median value of the ordered sequence arranged in the third step as the grayscale value; Step5: Transfer the grayscale value calculated in the fourth step to the pixel at the window center position.
[0044] Step S3: Based on the deep learning framework PyTorch, input the training set data and the validation set data to train the convolutional neural network model and extract the feature vectors.
[0045] Step S4: Based on the feature vector, use the softmax function to output the classification probability and obtain the classification result of zooplankton;
[0046] It should be noted that, please refer to the appendix Figure 4 In this embodiment, based on the PyTorch framework, a new convolutional neural network model is constructed for image classification. The training data set and the validation data set are input into the convolutional neural network model for training. The first layer of this model is the convolutional layer, including 64 convolutional kernels of size 5*5. After the first layer is the batch normalization layer, followed by a convolutional layer, using 64 convolutional kernels of size 5*5 and the Relu activation function, then adding the batch normalization layer, the average pooling layer and the Dropout layer to help extract image features and reduce overfitting. Repeat this structure multiple times to increase the complexity of the model and extract feature information. Then add a flattening layer to convert the feature map output by the previous convolutional layer into a one-dimensional vector. Then, add the batch normalization layer, the fully connected layer (with 128 neurons and the ReLU activation function), the activation layer and the Dropout layer in sequence. The last layer is the probability distribution output layer of the classification label, that is, the classification layer. Assuming we have 10 classification labels, use the softmax activation function to obtain the probability distribution result of zooplankton. Finally, use the test set data to evaluate the trained detection model.
[0047] Step S4 further includes: Based on the improved object detection network YOLOv7, localize and count the preprocessed image data to obtain the counting result of each type of zooplankton;
[0048] Specifically, the YOLOv7 algorithm model is developed by the YOLOv4 team and is a high-performance object detection model. Compared with most existing object detection algorithms, it has a faster detection speed and higher accuracy. The model includes four main components: image input, backbone network, neck network and head network. Among them, the backbone network integrates the BConv, ELAN and MP-1 convolutional modules, the neck network contains the ELAN-W and MP-2 modules, and replaces the SPP module with the SPPCSPC module to adapt to different input sizes. The head network adopts the RepConv module.
[0049] In this embodiment, a lightweight network based on the improved YOLOv7 is proposed. The attention mechanism SimAM is introduced into the Neck network of YOLOv7, which helps the model focus on key features without adding extra parameters and further improves the model's accuracy. Finally, the WIoU loss function is also adopted to achieve fast convergence. The information extracted by the BackBone network contains both effective features and invalid redundant information. Inserting the attention mechanism can pay more attention to the effective features. However, inserting the attention mechanism in the BackBone network may compress the space and channels of the feature map, resulting in a decline in the model's performance. Generally, the BackBone network of the model is not chosen to be damaged. Therefore, in this patent, the attention mechanism is inserted into the Neck network of the model. After obtaining the effective feature layer from the BackBone network, the attention mechanism is inserted for the first feature extraction. Then, after only fusing the features of the Neck network, it may not be able to effectively reduce the impact of unnecessary information on the model's detection ability. Therefore, the attention mechanism is inserted again after the output of the Neck network for the second feature extraction, which can give more priority to important features to improve the accuracy of model detection. The attention mechanism can assign greater weights to important features, reduce the weights of irrelevant information, thereby reducing the impact of redundant features, enhancing the ability to distinguish subtle differences, and improving the model's accuracy. The attention mechanism allows the model to use computing resources more effectively. By focusing on specific features, unnecessary computations are reduced, which is very crucial for lightweight models.
[0050] In addition, to achieve a faster convergence speed and more accurate target localization prediction, this study adopted the WIoU-v3 loss function for bounding box regression. WIoU-v3 introduced an innovative dynamic non-monotonic focusing mechanism that helps the model converge faster and improve training efficiency by using the outlier degree to evaluate the quality of anchor boxes.
[0051] Step S5: Integrate the classification result and the counting result, and visually output the integrated result.
[0052] Specifically, in Python, you can use the matplotlib library for visualizing the results. Assume that your classification results are stored in the classification_results list and the counting results are stored in the counting_results list. By integrating the code, you can achieve visualizing the results using the matplotlib library in Python. Assume that your classification results are stored in the classification_results list and the counting results are stored in the counting_results list. Use matplotlib to create a bar chart where the x-axis is the classification results and the y-axis is the corresponding counts. If your classification results are continuous numerical values, you may need to use different visualization methods such as scatter plots or histograms.
[0053] In this embodiment, the image data of the zooplankton is obtained by taking pictures of the zooplankton samples at different depths and positions; the image data is preprocessed to obtain the preprocessed image data, and the data is divided into training set data, validation set data, and test set data according to a preset ratio; the improved convolutional neural network model is trained by inputting the training set data and the validation set data to extract feature vectors; the softmax function is used to output the classification probability to obtain the classification results of the zooplankton; based on the improved object detection network YOLOv7, the preprocessed image data is located and counted to obtain the counting results of each type of zooplankton; the classification results and the counting results are integrated and then visualized and output. It improves the recognition and classification ability of zooplankton and reduces the requirements for hardware. In addition, it is presented to the user in a graphical way, which is convenient for the user to understand and analyze.
[0054] In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which a program for classifying and counting zooplankton based on deep learning is stored. When the program for classifying and counting zooplankton based on deep learning is executed by a processor, the steps of the method for classifying and counting zooplankton based on deep learning as described above are implemented.
[0055] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0056] The above computer-readable storage medium can be included in the deep learning-based zooplankton classification and counting device; it can also exist independently without being assembled into the deep learning-based zooplankton classification and counting analysis device.
[0057] In addition, an embodiment of the present invention also proposes a computer program product, including a deep learning-based zooplankton classification and counting program. When the deep learning-based zooplankton classification and counting program is executed by a processor, it implements the steps of the deep learning-based zooplankton classification and counting method described above.
[0058] The specific implementation manner of the computer program product of the present invention is basically the same as that of each embodiment of the above deep learning-based zooplankton classification and counting method, and will not be elaborated here.
[0059] Refer to Figure 5 , Figure 5An embodiment of the present invention also provides a zooplankton classification and counting system based on deep learning, characterized in that the system includes: an image data acquisition module for taking pictures of the zooplankton samples at different depths and positions to obtain the image data of the zooplankton; an image preprocessing module for preprocessing the image data to obtain the preprocessed image data, and dividing the preprocessed image data into training set data, validation set data, and test set data according to a preset ratio; a feature vector extraction module for training a convolutional neural network (CNN) based on the deep learning framework PyTorch by inputting the training set data and the validation set data, and extracting feature vectors; a classification module for outputting classification probabilities using the softmax function based on the feature vectors to obtain the classification results of the zooplankton; a counting module for positioning and counting the preprocessed image data based on the improved object detection network YOLOv7 to obtain the counting results of each type of zooplankton; and an output module for integrating the classification results and the counting results, and visually outputting the integrated results.
[0060] The zooplankton classification and counting system based on deep learning provided in this application adopts the zooplankton classification and counting system based on deep learning in the above embodiment, and can solve the technical problems of inaccurate and slow classification and recognition of zooplankton in the prior art. Compared with the prior art, the beneficial effects of the zooplankton classification and counting system based on deep learning provided in this application are the same as those of the zooplankton classification and counting method based on deep learning provided in the above embodiment, and other technical features in the zooplankton classification and counting system based on deep learning are the same as those disclosed in the above embodiment method, and will not be elaborated here.
[0061] It should be understood that the above is only for illustration and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can set according to needs, and the present invention does not limit this.
[0062] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no limitation is made here.
[0063] In addition, for the technical details not described in detail in this embodiment, reference can be made to the zooplankton classification and counting method based on deep learning provided in any embodiment of the present invention, and details will not be repeated here.
[0064] It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or system including such element.
[0065] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.
[0066] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory / random access memory, magnetic disk, optical disk), and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0067] The above are only the preferred embodiments of the present invention and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.
Claims
1. A zooplankton classification and counting method based on deep learning, characterized in that: The method comprises the following steps: S1. photographing the zooplankton sample at different depths and positions to obtain image data of the zooplankton; S2, preprocessing the image data to obtain preprocessed image data, and dividing the data into training set data, verification set data and test set data according to a preset ratio; S3. Based on the deep learning framework PyTorch, input the training set data and the validation set data to train the convolutional neural network model and extract feature vectors; S4, based on the feature vector, using the softmax function to output the classification probability to obtain the classification result of zooplankton; and based on the improved target detection network YOLOv7, positioning and counting the preprocessed image data to obtain the counting result of each zooplankton; S5. Integrate the classification result and the counting result, and output the integrated result in a visual format.
2. The deep learning-based zooplankton classification and counting method as described in claim 1 is characterized in that a high-definition CMOS camera and a trinocular microscope are used to photograph and collect the zooplankton samples at different depths and positions to obtain image data of the zooplankton.
3. The zooplankton classification and counting method based on deep learning according to claim 1, characterized in that: The preprocessing includes: labeling, resizing, graying, contrast enhancement and image denoising of the image data.
4. The zooplankton classification and counting method based on deep learning as claimed in claim 3, characterized in that: Use the preset image annotation tool LabelImg or Vott to annotate data and obtain the annotated image data; Resizing the labeled image data, wherein the resizing includes setting the size of the cropped sub-image to be within a range of 300-400 pixels in length and width; Gray-scaling the sub-image by using a weighted average method to obtain a gray-scaling sub-image; Performing contrast enhancement on the grayscale sub-image by using a histogram equalization method to obtain a contrast-enhanced sub-image; The contrast-enhanced sub-image is subjected to image denoising by using a median filtering method to obtain denoised image data.
5. The zooplankton classification and counting method based on deep learning according to claim 1, characterized in that: The convolutional neural network model includes a batch normalization layer, multiple convolutional layers, multiple average pooling layers, a Dropout layer, a flattening layer, a fully connected layer and an output layer; wherein the first layer is a convolutional layer, the second layer is a batch normalization layer, followed by a convolutional layer and a batch normalization layer, and then an average pooling layer and a Dropout layer in sequence, the above pattern is repeated multiple times, and finally a flattening layer is added, and a batch normalization layer, a fully connected layer and an output layer are added after the flattening layer, wherein the convolutional layer contains 64 5*5 convolution kernels, and the activation function adopts ReLU.
6. The zooplankton classification and counting method based on deep learning according to claim 1, characterized in that: The improved target detection network YOLOv5 includes four main components: image input, backbone network, neck network and head network. The backbone network integrates BConv, ELAN and MP-1 convolution modules, the neck network contains ELAN-W and MP-2 modules, and the SPP module is replaced by SPPCSPC module to adapt to different input sizes. The head network adopts RepConv module.
7. The method for zooplankton classification and counting based on deep learning according to claim 6, characterized in that: An attention mechanism SimAm is introduced into the neck network, and the WIoU loss function is adopted to achieve fast convergence.
8. A zooplankton classification and counting system based on deep learning, characterized in that: The system comprises: An image data acquisition module, used for taking photos of the zooplankton samples at different depths and positions to obtain image data of the zooplankton; An image preprocessing module, preprocessing the image data to obtain preprocessed image data, and dividing the preprocessed image data into training set data, verification set data and test set data according to a preset ratio; A feature vector extraction module, based on the deep learning framework PyTorch, inputs the training set data and the validation set data to train a convolutional neural network (CNN) and extract feature vectors; A classification module, based on the feature vector, uses a softmax function to output classification probabilities to obtain classification results of zooplankton; The counting module locates and counts the preprocessed image data based on the predetermined target detection network YOLOv7 to obtain the counting results of each zooplankton; The visualization module integrates the classification result and the counting result, and outputs the integrated result in a visualization manner.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the deep learning-based method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a deep learning-based zooplankton classification and counting program, which, when executed by a processor, implements the deep learning-based zooplankton classification and counting method as described in any one of claims 1 to 7.