Method for marine plankton identification based on in-situ images and deep learning

Abstract

The application discloses a marine plankton identification method based on in-situ images and deep learning, which constructs a multi-stage deep learning architecture in series for processing of full in-situ images, and comprises the following steps: a model is constructed based on a YOLOv8 skeleton to quickly locate and frame plankton individuals in a complex background; a U-Net model is used to finely segment the individuals in the in-situ images at a pixel level, extract accurate contours and obtain high-quality single images; finally, the images are input into a PlanktonNet network based on an innovative design of a ViT architecture, the self-adaptive small-size slice embedding and the introduction of a convolutional word embedding layer are adopted to improve the small-scale target feature extraction capability, and high-precision and fine-grained classification of genera and species is realized. The application has the advantages that the target detection, semantic segmentation and identification tasks are organically integrated, the problems of low identification efficiency and low automation degree in the prior art are effectively solved, and large batches of marine plankton can be quickly and accurately identified from in-situ images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical fields of deep learning, image processing and computer vision BACKGROUND

[0002] Marine plankton, as a key component of aquatic ecosystems, is highly sensitive to environmental changes and is an important indicator organism for monitoring the health of marine ecosystems. Traditional plankton observation relies mainly on physical methods such as net sampling, pump sampling or bottle sampling to obtain samples on site, and then the samples need to be taken back to the laboratory for manual counting and classification by professionals using microscopes. However, this "offshore post-processing" mode has obvious spatio-temporal lag, and the sample analysis period is long, the labor cost is high, and it is heavily dependent on expert experience, making it difficult to meet the urgent needs of modern marine observation for high spatio-temporal resolution and real-time performance.

[0003] With the rapid development of underwater optical imaging technology, various in-situ microscopic imaging instruments (such as UVP6, etc.) have been widely deployed in marine observation networks. These instruments can obtain massive in-situ microscopic images without damaging the living environment of organisms. Unlike slide images in a laboratory environment, in-situ images often face challenges such as complex background (presence of suspended particles, uneven lighting), variable target organism posture and overlapping. Existing image processing techniques often struggle to balance efficiency and accuracy when dealing with this specific type of data. Therefore, it is urgent to develop an automatic and intelligent analysis method specifically for marine plankton in-situ images to solve the bottleneck of "easy to obtain, difficult to process" massive image data, and to achieve rapid and accurate monitoring of plankton populations, thereby improving the efficiency of marine plankton population research. SUMMARY

[0004] Therefore, the present application provides a marine plankton identification method based on in-situ images and deep learning, which aims to solve the above technical problems. The specific technical method of the application comprises the following steps:

[0005] A marine plankton identification method based on in-situ images and deep learning, characterized by comprising the following steps:

[0006] S1: Collecting in-situ observation images of plankton;

[0007] S2: Preprocessing the images, manually framing and positioning the plankton targets in the images, and selecting the genus and species categories to generate training samples containing plankton bounding box position coordinates and plankton classification labels, and generating training set and validation set for subsequent model training;

[0008] S3: Construct a target detection model, improve based on YOLOv8 architecture, add a high-resolution P2 detection layer based on the original P3, P4 and P5 detection layers, embed an attention mechanism module at the connection between the backbone network and the neck network, construct an improved YOLOv8 target detection model, use the shallow network features to improve the detection capability of microzooplankton, and use the training set of step S2 to train, output the bounding box containing the position coordinates and the confidence, and according to the bounding box coordinates, cut out each zooplankton individual from the original image as the primary region of interest;

[0009] S4: Construct a semantic segmentation model, improve based on U-Net architecture, construct an AttentionRes-U-Net model, which uses residual modules instead of traditional convolutional layers in the encoder and decoder, and transmits features through skip connections; input the primary region of interest obtained in step S3 into the model for training to generate single zooplankton images without background; manually mark and identify the obtained single zooplankton images to form a data set for subsequent training of the image recognition network PlanktonNet;

[0010] S5: Construct an image recognition model, design a PlanktonNet network model based on ViT architecture, which introduces a convolutional word embedding layer, discards the traditional ViT encoding method of directly flattening and linearly projecting the image blocks, constructs a convolutional word embedding layer at the input end, and uses a multi-layer convolutional neural network CNN to extract and encode map the micro image blocks; through the overlapping sliding and parameter sharing mechanism of the convolution kernel, local induction bias is introduced for the input sequence to strengthen the model's ability to capture local texture, edge direction and spatial neighborhood information of zooplankton, and generate visual vector sequences rich in local semantic information; process the single zooplankton images output by step S4 to output zooplankton genus and species identification results, and save the model parameters with the highest recognition accuracy on the validation set during the training process as the final classification model;

[0011] S6: Input the real-time collected marine zooplankton in-situ images into the trained YOLOv8 target detection model, U-Net semantic segmentation model and PlanktonNet image recognition model in turn, and automatically output the final recognition results containing individual position, contour and genus and species information.

[0012] In step S1, the in-situ observation images of zooplankton are taken by in-situ observation instruments on the sea observation platform, and the original in-situ images are reconstructed at multiple scales using image pyramid technology to generate an image set containing different resolution levels to construct a data representation of multi-scale features.

[0013] In the step S2, the original image collected in the step S1 is subjected to quality enhancement and super-resolution reconstruction, specifically including filtering out suspended particle noise in the water body by using a denoising algorithm, improving the target contrast by using an image enhancement technique, and restoring the texture details of microzooplankton by using a super-resolution calculation technique; the processed image is subjected to normalization processing, so that the gray scale distribution of the entire zooplankton image dataset meets the standard normal distribution with a mean of 0 and a standard deviation of 1 through pixel-level conversion; a true value dataset is constructed based on the above high-quality normalized image, the zooplankton target is manually framed and positioned and the genus and species categories are selected by using an image labeling tool LabelImg, and a training sample containing accurate position coordinates and classification labels is generated; the above processed image is uniformly imported into a standardized data warehouse and subjected to cleaning and format checking; then, the cleaned dataset is randomly divided into a training set, a validation set and a test set in a ratio of 7:2:1, thereby providing a multi-scale and high-standard data basis for subsequent model training and evaluation.

[0014] In the step S3, in view of the characteristics of marine zooplankton that the proportion in the in-situ image is small and it is easy to be confused with the background noise of the water body, a specially optimized YOLOv8 target detection model is constructed. First, the general YOLOv8 network structure is improved, and a high-resolution P2 detection layer is added on the basis of the original P3, P4 and P5 detection layers, so as to use the rich spatial geometric features retained by the shallow network to improve the perception ability of microzooplankton and relieve the feature loss problem caused by multiple down-sampling; an attention mechanism module is embedded at the connection between the backbone network Backbone and the neck network Neck, so as to give the model the ability to focus on the morphological features of zooplankton and suppress the background interference weight; the training set generated in the step S2 is used to train the improved model, and the loss function is optimized to improve the boundary box regression accuracy; and in the inference stage, the entire in-situ microscopic image to be processed is input into the trained model for global scanning, and the boundary box containing coordinate information and the confidence are output, and the individual is cut out from the original image according to the above, as a primary region of interest for the subsequent segmentation network.

[0015] In the step S3, in the inference stage, after the individual is cut out according to the output boundary box, the cut-out image is subjected to image augmentation operations such as rotation, flipping or scaling to increase sample diversity and prevent overfitting of the cutting model.

[0016] In step S4, an attention gate mechanism is embedded at the transverse skip connection of the encoder and the decoder, which can automatically suppress the water background response weight in the feature map and focus on the saliency regions such as the translucent edge and fine tentacle of the plankton, so as to accurately capture the weak boundary information; the primary region of interest output in step S3 is input into the model, which is supervised and trained by a hybrid loss function, and finally a high-precision binary mask is generated; the original image is subjected to matrix operation by using the mask, so as to completely remove the background noise and extract a pure plankton image retaining complete edge and texture information.

[0017] In step S4, the model is supervised and trained by using a hybrid loss function, which is a weighted combination of Dice Loss and Cross-Entropy Loss; the model finally generates a binary mask, and the background noise is removed by matrix operation.

[0018] In step S5, the PlanktonNet network model is constructed by using an adaptive small-size slicing strategy, according to the scale characteristics of the preprocessed plankton image, the input image is divided into 4x4 or 8x8 pixel micro blocks.

[0019] In step S5, the high-resolution visual vector sequence embedded by convolution is input into the Transformer encoder, the long-distance dependence of the features in the whole image is calculated by using the multi-head self-attention mechanism, the global topological structure and morphological combination of the plankton are comprehensively understood, and the shortage of the limited receptive field of the convolution operation is made up; finally, the feature vector fused with the “local fine texture” and the “global morphological structure” is input into the classification head MLP, and the accurate species identification result of the plankton is output.

[0020] The application has the following advantages: the application adopts a three-level cascade architecture of "target positioning-fine segmentation-species identification", embodies the design idea of "each performs its own function and optimizes in coordination", and realizes complete intelligent monitoring from visual perception to ecological cognition. In view of the characteristics of complex background, small target and variable texture of marine in-situ microscopic images, the application introduces a high-resolution P2 layer at the detection end, fuses an attention mechanism at the segmentation end, and constructs a "adaptive convolution slice + Transformer" hybrid architecture (PlanktonNet) at the recognition end, effectively reduces the interference of water suspended particles and light noise, significantly improves the capture ability and fine-grained classification accuracy of small, transparent and irregular plankton, and the system can automatically output comprehensive monitoring results containing individual position, accurate contour and species attributes, overcoming the defects of insufficient feature extraction capability of existing single models in complex in-situ environment, greatly improving the accuracy and robustness of identification, and completing intelligent identification of marine plankton for in-situ microscopic images, which far exceeds the performance of single models or traditional methods. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used for the description of the embodiments.

[0022] Figure 1 A flowchart of a marine plankton identification method based on in-situ images and deep learning disclosed in the embodiments of the application;

[0023] Figure 2 A marine plankton in-situ image target detection backbone network structure diagram based on a Yolov8 skeleton structure disclosed in the embodiments of the application;

[0024] Figure 3 A marine plankton in-situ image semantic segmentation model structure diagram based on a U-net network structure disclosed in the embodiments of the application;

[0025] Figure 4 A PlanktonNet marine plankton in-situ image identification model structure diagram based on a ViT architecture disclosed in the embodiments of the application;

[0026] Figure 5 A technical roadmap of a marine plankton identification method based on in-situ images and deep learning disclosed in the embodiments of the application; DETAILED DESCRIPTION

[0027] The technical solutions of the embodiments of the application will be described below with reference to the drawings in the embodiments of the application. The described embodiments are only some of the embodiments of the application, not all the embodiments.

[0028] Reference Figure 1 The application discloses a marine plankton identification method based on in-situ images and deep learning, which comprises the following steps:

[0029] S1: Collecting marine plankton in-situ images by relying on marine in-situ microscopic observation instruments;

[0030] S2: Preprocessing the plankton in-situ images, including image denoising, image enhancement, super-resolution calculation and other operations; using Labellmg and other labeling tools to draw a frame, frame the plankton appearing in the in-situ image, and divide the data set according to the corresponding proportion;

[0031] S3: Building a target detection model through YOLOv8 architecture, inputting the whole in-situ microscopic image for training, the model outputting the boundary box and corresponding confidence of the plankton individual, and cutting the image according to the boundary box to form a training set for the subsequent training segmentation model;

[0032] S4: Building a semantic segmentation model through U-Net model, performing pixel-by-pixel classification on the input cropped image, and the model outputting high-quality single plankton image without background, and researchers identifying and classifying the single plankton image to form a high-quality marine plankton image data set;

[0033] S5: Designing a PlanktonNet model based on ViT architecture, inputting the single plankton image for training, and the model outputting the marine plankton identification result;

[0034] S6: Inputting the real-time photographed and collected marine plankton in-situ microscopic image into the trained deep learning model to obtain the identified result image, and realizing the identification of the marine plankton.

[0035] In specific embodiments, the S1 step aims to build a plankton basic database covering multiple sources, multiple scales and high standardization, specifically including:

[0036] S11: Collecting standard images of plankton in a laboratory environment by using a microscopic imaging system, and simultaneously collecting in-situ images of plankton in a marine environment by using in-situ microscopic observation instruments (UVP6, etc.) deployed on a marine observation platform, and taking the image data collected by the in-situ microscopic observation instruments as the main part. The in-situ images contain complex background noise (such as sea snow, suspended particulate matter), which is used to improve the generalization ability of the model.

[0037] S12: According to the characteristics of the great individual differences (micron to centimeter) of plankton, the collected original images are subjected to downsampling and resampling processing, and a series of image sets with different resolutions are generated by using the image pyramid technology. This step ensures that the model can capture morphological features at different scales.

[0038] S13: Convert the above multi-source, multi-scale images into a standard format (such as.jpg or.png) and integrate them into a unified distributed data warehouse. Each image is uniquely numbered and its collection environment parameters (such as salinity, depth, illumination) are recorded to form structured metadata.

[0039] S2: Improve image quality through algorithmic means and provide true value labels for supervised learning to build a dataset for deep learning model training. The specific steps include:

[0040] S21: For the haze, motion blur and background interference in the in-situ images, use median filtering or Gaussian filtering for preliminary denoising. Further use the limited contrast adaptive histogram equalization technique to enhance the contrast of the edge and internal texture of the plankton. For blurred images, introduce super-resolution reconstruction algorithms (such as SRCNN or GAN-based methods) to improve image clarity.

[0041] S22: To eliminate differences caused by different lighting conditions and sensor sensitivity, perform normalization on all images. By calculating the mean and standard deviation of the entire dataset, adjust the image pixel values to a normal distribution with a mean of 0 and a standard deviation of 1. This speeds up the convergence process of the deep learning model and improves training stability.

[0042] S23: Use annotation tools (such as LabelImg) to annotate the target detection box of the plankton individuals in the image, record their class name and center point coordinates, width and height, and generate corresponding XML or TXT format label files.

[0043] S24: To prevent model overfitting, dynamically expand the training set before training, including but not limited to: horizontal / vertical flipping, rotation, random cropping, brightness adjustment and adding simulated random noise. Through data augmentation, the variability of the marine environment is simulated to make the model more robust. Random stratified sampling method is used to divide the images in the data warehouse into training set, validation set and test set in the ratio of 7:2:1. The training set is used for model parameter learning, the validation set is used for hyperparameter tuning and model selection, and the test set is used to evaluate the performance indicators of the final recognition method.

[0044] In this embodiment, S3 is the key step to realize the transition from "large field of view in-situ image" to "individual region of interest". S3 realizes individual positioning in complex background by building and training a YOLOv8 model optimized for micro targets. The specific operation steps are as follows:

[0045] ​S31: In order to solve the problem that plankton occupies a very small proportion in the in-situ image and is easily confused with "sea snow", the embodiment deeply customizes the YOLOv8 network: refer to Figure 2 On the basis of the original P3, P4 and P5 detection layers, a P2 high-resolution detection layer is added by up-sampling and fusing shallow layer features. This layer retains more spatial geometric information, significantly improving the perception accuracy of micron-level plankton individuals. An attention mechanism module is embedded at the connection between the backbone network and the neck network, which automatically suppresses the weights of background noise such as bubbles and suspended particles through global context modeling, allowing the model to focus on the morphology and texture of plankton. The C2f module is used to replace the traditional C3 module. After the input feature map is processed by the convolution layer (Conv+BN+SiLU), it is divided into two branches by channel. One branch performs multi-scale feature extraction through a list composed of multiple Bottleneck. If the input and output channels are consistent and shortcut is True, residual connection is used. The activation function uses SiLU, whose formula is:

[0046]

[0047] wherein, is the input value.

[0048] The Head module adopts a bidirectional feature pyramid structure from bottom to top and from top to bottom, fully fusing deep semantic and shallow positioning information. The classification and positioning tasks are separated, and each detection head contains an independent convolution sequence (Conv+Conv+nn.Conv2d), which outputs class probability and bounding box coordinates respectively. The traditional anchor matching mode is abandoned, and the center point and height of the plankton are directly predicted. This design reduces the redundancy of hyperparameters, effectively improving the computational efficiency and detection accuracy in dense small target scenarios.

[0049] S32: In the training process, the TaskAlignedAssigner positive sample assignment strategy is used, and the composite loss function is used for parameter optimization. The total loss value is composed of the following three parts:

[0050] Classification loss (BCE Loss): binary cross-entropy loss is used, which supports multi-label classification, and the formula is:

[0051]

[0052] wherein, is the actual label of the sample, is the probability of the model predicting the sample (i.e. the label is 1).

[0053] CIoU Loss: The CIoU loss considers the overlap area, the Euclidean distance between the center points, and the consistency of the aspect ratio:

[0054]

[0055] wherein: is the overlap area, is the Euclidean distance between the center points of the predicted box and the ground truth box, c is the length of the diagonal of the minimum bounding rectangle of the two boxes, is the aspect ratio consistency term, is the weight coefficient, is the width of the ground truth box, is the height of the ground truth box, is the width of the predicted box, is the height of the predicted box.

[0056] DFL: For small targets with fuzzy boundaries, the coordinates are modeled as a probability distribution, and the regression stability is improved by optimizing the distribution shape. Assuming that the true value of the target coordinate is between and , and , the DFL calculation formula is:

[0057]

[0058] wherein: is the true label, and are adjacent discrete labels, and are the corresponding probabilities predicted by the model.

[0059] S33: Input the in-situ microscopic image to be processed into the improved YOLOv8 model trained, and output all the bounding boxes with confidence that meets the standard. According to the coordinate information, the plankton monomer image is cropped from the high-resolution original image as the primary region of interest (ROI). To prevent overfitting of the subsequent S4 segmentation model, the cropped ROI image is subjected to secondary enhancement processing, including random angle rotation, horizontal / vertical flipping, and scaling offset. This step aims to increase sample diversity and ensure that the model can cope with the random distribution of plankton in water.

[0060] In the specific embodiments of the present application, as Figure 3 , the S4 step is the core of realizing the transition from "rough positioning" to "pixel-level extraction", and provides pure morphological features for the final classification by eliminating complex background noise. The S4 step uses a deep convolutional neural network to perform fine processing on the primary region of interest (ROI) cut by S3, and the specific steps are as follows:

[0061] S41: In view of the characteristics of high resolution and fine edge texture of the in-situ image of zooplankton, the embodiment adopts Overlap-tile (tiling strategy) to process the input image. In the embodiment, the original ROI image of 512*512 pixels can be expanded to 696*696 pixels through mirror padding. According to the tiling logic, four overlapping sub-images of 572*572 are cut out as input, so that the edge area can also obtain complete context information, and finally a seamless 388*388 prediction map is output.

[0062] S42: In the left shrinkage path of U-Net, features are extracted through repeated convolution structures. Each layer adds a residual module to alleviate the gradient vanishing problem of deep network through a jump connection. Each level contains two 3*3 non-padding convolutions and a ReLU activation function, followed by a 2*2 max pooling layer (Stride=2) for down sampling. The convolution output size formula is:

[0063] .

[0064] wherein, represents the size of the input image, represents the size of the convolution kernel, represents the size of the padding, represents the size of the stride, represents the size of the output image, and is rounded down when it is not an integer. In this step, the input is 572*572, the convolution is 3*3, and there are 64 convolution blocks. .

[0065] S43: In the expansion path on the right side of U-Net, the dimension of the feature map is restored and accurately reconstructed. Deconvolution is used for upsampling, and the feature map size is doubled and the channel number is halved at each step. An attention module is embedded at the lateral jump connection. This module weights the low-level feature maps from the encoder, automatically suppresses the sea snow and bubble response in the water background, and enhances the perception of key areas such as the translucent plankton edge and tentacles. The feature map enhanced by attention is spliced with the decoder features, and after two 3*3 convolutions, the last 1*1 convolution maps the 64-channel feature map to a binary classification result (foreground / background).

[0066] S44: In order to solve the problems of individual contact, overlap and class imbalance of zooplankton, the embodiment adopts a weighted cross-entropy loss function for optimization:

[0067] The weight map is calculated in advance Morphological operations are used to identify the boundaries of adjacent individuals, increasing the loss weight of pixels in contact areas to force the network to learn extremely small segmentation boundaries. This is combined with pixel-wise softmax and cross-entropy, and the energy function... The formula is:

[0068]

[0069] in, It is the softmax loss function. It is a collection of pixel label values. It is the pixel weight. It's about vectors. A predefined set of weight functions, It is a constant term representing the basic weights. It is the exponential part of the Gaussian function, where and It's about vectors. Two distance functions.

[0070] The model employs stochastic gradient descent (SGD) with a high momentum coefficient (0.99). With large image patch inputs, even with a batch size of 1, historical gradients can be accumulated through the momentum term, ensuring the model's training stability on high-resolution tasks.

[0071] S45: The pixel-level prediction results output by U-Net are thresholded to generate a high-precision binary mask. This mask is then used to perform a matrix multiplication operation with the original ROI map to completely remove background noise from the water body. The final extracted images of individual zooplankton retain complete outlines, bristles, and internal semi-transparent structures. The researchers then labeled and classified these images to form a training set, providing standard input for the genus and species classification in the subsequent S5 step.

[0072] In the specific implementation method of the present invention, reference is made to... Figure 4 Step S5 is the core step in realizing the transformation from "image features" to "classification conclusions." This embodiment constructs a PlanktonNet network model by improving the ViT architecture, aiming to improve the recognition accuracy of highly similar marine zooplankton through refined local feature capture and global morphological modeling. The specific steps are as follows:

[0073] S51: For the characteristics of plankton individual micro and limb details (such as the setae of copepods, limb joints) which are crucial for identification, this embodiment reconstructs the input high-quality single image: fine-grained slicing strategy: abandon the large size slice of 16*16 pixels adopted by traditional ViT, this embodiment adopts an adaptive small size slicing strategy. The input image (such as 224*224*3) is divided into 4*4 or 8*8 small patches, significantly increasing the sequence length N to ensure that high-frequency morphological details are not lost. A multi-layer convolutional neural network (CNN) is constructed at the input end instead of the traditional linear projection. The overlapping sliding and parameter sharing characteristics of the convolution kernel are used to extract spatial information within and between adjacent slices. This step introduces local inductive bias to the visual sequence, enhancing the model's ability to capture plankton edge direction and local texture.

[0074] S52: After convolutional embedding, a high-dimensional visual vector sequence with a dimension of N*768 is generated. A learnable special classification character [CLS] is added at the beginning of the input sequence to collect the feature representation of the whole image, and the final sequence dimension becomes (N+1)*768. To make up for the lack of spatial position awareness of the Transformer architecture, a set of standard learnable one-dimensional position encodings is superimposed on the feature vector. The encoding has the same dimension as the input vector, ensuring that the model can learn the relative position of plankton organs and limbs in the overall topological structure.

[0075] S53: The constructed vector sequence is sent to the stacked Transformer Encoder module, and each Block performs the following operations: first, map the input to q, k, v vectors. The embodiment uses 12 attention heads, each of which is calculated in parallel on a 197*64 dimension. The global attention mechanism is used to obtain long-range dependencies in the image and comprehensively understand the global morphological composition of plankton. Its formula is expressed as:

[0076]

[0077] Where Q (Query) is the query matrix, K (Key) is the key matrix, V (Value) is the value matrix, and d is the dimension of the key vector.

[0078] S54: After the alternative processing of multiple Encoder Blocks, the output vector corresponding to the [CLS] character is extracted as the feature table of the whole image. The vector is input into the classification head (MLP) composed of fully connected layers. The MLP first enlarges the feature dimension (e.g., from 768 to 3072), and then reduces and maps it to the target number of species categories. Cross-entropy loss function is used for supervised learning. During the training process, the recognition accuracy on the validation set is monitored in real time, and the model weight with the highest recognition rate is saved as the final intelligent plankton recognition model.

[0079] Step S6: Refer to Figure 5 The in-situ microscopic image collected by the offshore observation platform in real time is sequentially subjected to positioning and cutting in S3, fine segmentation in S4, and PlanktonNet recognition in S5, and finally an identification result image containing individual position frame, accurate contour edge, and species label is automatically output, realizing intelligent real-time monitoring of marine plankton populations.

[0080] The intelligent plankton recognition system based on in-situ microscopic images provided in the present application adopts the intelligent plankton recognition method based on in-situ microscopic images in the above embodiments, which can solve the technical problems of inaccurate recognition of marine plankton in an in-situ environment, easy loss of features of small individuals, and low efficiency of in-situ monitoring in the prior art. Through the synergistic effect of steps S1 to S5, the present application uses multi-scale reconstruction, deep target positioning, accurate edge segmentation, and image recognition technology that fuses local and global features, significantly improving the recognition accuracy and automation level of plankton in complex sea conditions. Compared with the prior art, the intelligent plankton recognition system based on in-situ microscopic images provided in the present application has the same beneficial effects as the intelligent plankton recognition method based on in-situ microscopic images provided in the above embodiments, and other technical features in the recognition system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0081] It should be understood that the above is only for illustration, and does not constitute any limitation on the technical solutions of the present application. In specific applications, those skilled in the art can set it up as needed, and the present application does not limit it. It should be noted that the above-described workflow is only illustrative and does not limit the scope of protection of the present application. In actual application, those skilled in the art can select part or all of them to achieve the purpose of the embodiment scheme according to actual needs, which is not limited here.

[0082] The above merely describes the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application, and any equivalent structure or equivalent process conversion, or direct or indirect application in other related technical fields, which are made by using the content of the present application specification and drawings, are also included in the patent protection scope of the present application.

Claims

1. A marine plankton identification method based on in-situ images and deep learning, characterized by, The method comprises the following steps: S1: collecting in-situ observation images of zooplankton; S2: preprocessing the images, manually framing and positioning the zooplankton targets in the images, and selecting the genus and species categories to generate training samples containing the position coordinates of the zooplankton bounding boxes and the zooplankton classification labels, and generating a training set and a validation set for subsequent model training; S3: constructing a target detection model, improving the YOLOv8 architecture, adding a high-resolution P2 detection layer based on the original P3, P4 and P5 detection layers, embedding an attention mechanism module at the connection between the backbone network Backbone and the neck network Neck, constructing an improved YOLOv8 target detection model, and using the shallow network features to improve the detection capability of microzooplankton; and training the training set of step S2 to output the bounding box containing the position coordinates and the confidence, and cutting out each zooplankton individual from the original image as a primary region of interest according to the bounding box coordinates; S4: constructing a semantic segmentation model, improving the U-Net architecture, constructing an AttentionRes-U-Net model, which uses a residual module to replace the traditional convolutional layer in the encoder and decoder, embeds an attention gate mechanism at the horizontal skip connection Skip Connection of the encoder and decoder, and transmits the features through the skip connection; The primary region of interest obtained in step S3 is input into the model for training to generate a single zooplankton image without background; the single zooplankton image obtained is manually labeled and identified to form a data set for training the subsequent image recognition network PlanktonNet; S5: constructing an image recognition model, designing a PlanktonNet network model based on the ViT architecture, introducing a convolutional word embedding layer, abandoning the traditional ViT encoding method of directly flattening and linearly projecting the image blocks, constructing a convolutional word embedding layer at the input end, and using a multi-layer convolutional neural network CNN to extract and encode map the features of the micro image blocks; through the overlapping sliding and parameter sharing mechanism of the convolution kernel, a local induction bias is introduced for the input sequence to strengthen the model's ability to capture local texture, edge direction and spatial neighborhood information of zooplankton, and generate a visual vector sequence rich in local semantic information; the single zooplankton image output by step S4 is processed to output the zooplankton genus and species identification result, and in the training process, the model parameters with the highest identification accuracy on the validation set are saved as the final classification model; S6: input the real-time collected in-situ images of marine zooplankton into the trained YOLOv8 target detection model, U-Net semantic segmentation model and PlanktonNet image recognition model in sequence, and automatically output the final identification result containing the individual position, contour and genus and species information. 2.The marine plankton identification method based on in-situ images and deep learning according to claim 1, characterized in that, In the step S1, the in-situ observation image of the zooplankton is shot by using the in-situ observation instrument on the offshore observation platform, and the original in-situ image is reconstructed in multiple scales by using the image pyramid technology to generate an image set containing different resolution levels, so as to construct a data expression of multi-scale features. 3.The method of claim 1, wherein, In the step S2, the original image collected in the step S1 is subjected to quality enhancement and super-resolution reconstruction, specifically including filtering the suspended particle noise of the water body by using a denoising algorithm, improving the target contrast by using an image enhancement technology, and restoring the texture details of the microzooplankton by using a super-resolution calculation technology; the processed image is subjected to normalization processing, so that the gray scale distribution of the entire zooplankton image dataset meets the standard normal distribution with a mean of 0 and a standard deviation of 1 through pixel-level transformation; a true value dataset is constructed based on the normalized image, the zooplankton target is positioned and the species category is selected by manual framing by using an image labeling tool LabelImg, and a training sample containing accurate position coordinates and classification labels is generated; the processed image is uniformly imported into a standardized data warehouse, and is subjected to cleaning and format checking; then, the cleaned dataset is randomly divided into a training set, a validation set and a test set according to a ratio of 7:2:1, so as to provide a multi-scale and high-standard data basis for subsequent model training and evaluation.

4. The method of claim 1, wherein, In the step S3, in view of the characteristics that the marine zooplankton accounts for a small proportion in the in-situ image and is easily confused with the background noise of the water body, a specially optimized YOLOv8 target detection model is constructed, first, the general YOLOv8 network structure is improved, a high-resolution P2 detection layer is added based on the original P3, P4 and P5 detection layers, the rich spatial geometric features retained by the shallow network are used to improve the perception ability of the microzooplankton and relieve the feature loss problem caused by multiple down-sampling; an attention mechanism module is embedded at the connection between the backbone network Backbone and the neck network Neck, so as to give the model the ability to focus on the morphological features of the zooplankton and suppress the background interference weight; the training set generated in the step S2 is used to train the model, and the loss function is optimized to improve the boundary box regression accuracy; In the inference stage, the entire in-situ microscopic image to be processed is input into the trained model for global scanning, and the boundary box containing the coordinate information and the confidence are output, and the individual is cropped from the original image as a primary region of interest for the subsequent segmentation network.

5. The method of claim 4, wherein, In the step S3, in the inference stage, the cropped image is subjected to image augmentation operations such as rotation, flipping or scaling to increase the sample diversity and prevent overfitting of the cutting model. 6.The method of claim 1, wherein, The attention gate mechanism in the step S4 can automatically suppress the water background response weight in the feature map and focus on the saliency region of the plankton including the translucent edge and the fine tentacle, so as to accurately capture the weak boundary information; the primary region of interest output in the step S3 is input into the model, the model is supervised and trained through a hybrid loss function, and finally a high-precision binary mask is generated; the original image is subjected to matrix operation by using the mask, the background noise is completely removed, and a pure single plankton image retaining complete edge and texture information is extracted.

7. The method of claim 6, wherein, In the step S4, the model is supervised and trained by using a hybrid loss function, the hybrid loss function is a weighted combination of Dice Loss and Cross-Entropy Loss; the model finally generates a binary mask, and the background noise is removed through matrix operation. 8.The method of claim 1, wherein, In the step S5, the PlanktonNet network model is constructed, an adaptive small-size slice strategy is adopted, the input image is divided into 4*4 or 8*8 pixel micro blocks according to the scale characteristics of the preprocessed plankton image. 9.The marine plankton identification method based on in-situ images and deep learning of claim 1, wherein, In the step S5, the high-resolution visual vector sequence embedded through convolution is input into a Transformer encoder, a multi-head self-attention mechanism is used to calculate the long-distance dependence relationship of the features in the whole image, and the global topological structure and morphological combination of the plankton are comprehensively understood, so as to make up for the limitation of the receptive field of the convolution operation; finally, the feature vector fused with the "local fine texture" and the "global morphological structure" is input into a classification head MLP, and the accurate species identification result of the plankton is output.

Images