Enhanced YOLO-based health status early warning method for artificially cultivated shellfish larvae

Abstract

The invention discloses an enhanced YOLO-based artificial cultivation shellfish larva health state early warning method, and belongs to the field of ocean and biological breeding, and the method comprises the steps: obtaining a shellfish larva image, carrying out the preprocessing of the shellfish larva image, and obtaining a preprocessing data set; optimizing the YOLOv8 model based on a CG module and a context self-attention mechanism, and constructing a deep learning model; training the deep learning model through the preprocessed data set to obtain a larva health state recognition model; wherein a loss function of the deep learning model is an Inner-CIoU loss function; and identifying the health state of the larva based on the larva health state identification model. According to the method provided by the invention, the shellfish larvae can be effectively identified, the health states of the larvae can be accurately classified, and a feasible solution is provided for the health state early warning task of artificially cultivated shellfish larvae.

Description

Technical Field

[0001] The present invention belongs to the fields of ocean and biological breeding, and particularly relates to a method for warning the health status of artificially cultivated shellfish larvae based on enhanced YOLO. Background Art

[0002] Recently, with the progress of machine vision systems, object detection and counting methods based on image processing technology have been widely studied and applied. These methods not only reduce costs but are also easier to implement, showing great potential in improving efficiency and accuracy. In image processing technology, visible features of objects, including shape, size, color, and texture, are usually extracted through image analysis and processing, and then the objects are segmented from the background and counted based on one or several of the above features. These features, such as color and texture, are low-level features obtained through artificial feature extraction methods, which are very accurate for detecting and counting objects in a sparse state. However, shellfish larvae are very small in size, and often appear densely stacked and partially overlapped in samples, with little difference between individuals, especially having a high overall similarity in color and morphological characteristics. This situation poses great challenges to image processing and feature extraction.

[0003] Object detection algorithms based on deep learning can provide effective methods for detecting the health status of shellfish larvae. However, because shellfish larvae are very small in size, extremely similar in appearance, and the resolution of a single larva in the image is not high, the features available for differentiation are often limited to a very small part of the image, making it difficult for the model to learn these features. Therefore, the detector may miss some features, resulting in inaccurate detection results, which in turn affects the reliability of aquaculture management. Therefore, for the detection of the health status of shellfish larvae based on deep learning, the key is to be able to effectively extract and utilize the useful features of these local areas of shellfish larvae. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method for warning the health status of artificially cultivated shellfish larvae based on enhanced YOLO, including:

[0005] Obtain shellfish larva images, preprocess the shellfish larva images to obtain a preprocessed data set;

[0006] Optimize the YOLOv8 model based on the CG module and the context self-attention mechanism to construct a deep learning model;

[0007] Train the deep learning model with the preprocessed data set to obtain a larva health status recognition model; wherein, the loss function of the deep learning model is the Inner-CIoU loss function.

[0008] Identify the health status of the larvae based on the larval health status recognition model.

[0009] Preferably, the process of obtaining the preprocessed data set includes:

[0010] Obtain the shellfish larva image, manually annotate the shellfish larva image to obtain an annotated image;

[0011] Perform brightness transformation, multi-angle rotation, and noise addition operations on the annotated image in sequence to obtain an augmented data set;

[0012] Randomly divide the augmented data set to generate the preprocessed data set.

[0013] Preferably, the process of constructing the deep learning model includes:

[0014] Construct a CG module based on a local feature extractor, a surrounding context extractor, a joint feature extractor, and a global context extractor;

[0015] Construct a context self-attention mechanism module based on context information and self-attention mechanism;

[0016] Replace the original downsampling module of the YOLOv8 model with the CG module, and introduce the context self-attention mechanism module at the end of the backbone network of the YOLOv8 model to complete the construction of the deep learning model.

[0017] Preferably, the operation process of the CG module includes:

[0018] Learn local features in adjacent feature vectors through the local feature sensor;

[0019] Capture surrounding context information based on the surrounding context extractor;

[0020] Based on the joint feature extractor, fuse the local features and the surrounding context information, optimize through batch normalization and parametric ReLU activation function, then collect global context features through the global context extractor, extract global information through a multi-layer perceptron, and finally generate an output feature map through a weighted operation.

[0021] Preferably, the operation process of the context self-attention mechanism module includes:

[0022] Receive the input feature map, process all adjacent key vectors within the k×k grid in the input feature map based on k×k group convolution, and obtain static context information based on the adjacent key vectors;

[0023] Concatenate the static context information with the input query vector, and obtain the attention matrix through two 1×1 convolutions;

[0024] Multiply the attention matrix by the value vector processed by 1×1 convolution to obtain the global context information;

[0025] Aggregate the static context information and the global context information to complete self-attention learning.

[0026] Preferably, the process of optimizing the YOLOv8 model based on the CG module and the context self-attention mechanism further includes: integrating a small object detection head of 160×160 in the detection layer of the YOLOv8 model;

[0027] The operation process of the small object detection head includes: stacking the 80×80 scale feature map of the second layer of the backbone network with the upsampled feature layer in the neck network, and after passing through the C2f layer and upsampling, obtaining a feature layer with small object feature information. After splicing and fusing the feature layer with the small object feature information and the 160×160 feature map output by the second layer of the backbone network, the output is completed, and the detection of the small object feature is completed.

[0028] Preferably, the defined expression of the Inner-CIoU loss function is:

[0029]

[0030] Among them, is the Inner-CIoU loss function, is the CIoU loss function, IoU is the intersection of B and B gt where B is the predicted bounding box, B gt is the ground truth bounding box, IoU inner is the internal intersection over union.

[0031] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor. When the processor executes the computer program, the method is implemented.

[0032] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method is implemented.

[0033] Compared with the prior art, the present invention has the following advantages and technical effects:

[0034] (1) The present invention improves the downsampling convolutional module in the YOLOv8 backbone network and replaces it with a ContextGuided (CG) module. The CG module can simultaneously process local features, surrounding context, and global context, effectively fuse this feature information, thereby strengthening the model's ability to detect closely adhered or overlapping targets, enabling the model to more accurately identify and distinguish smaller targets with dense arrangements or complex backgrounds. This change not only expands the model's perspective range but also enhances its feature extraction ability.

[0035] (2) The present invention also introduces a contextual self-attention mechanism called Contextual Transformer (CoT), which is combined with the C2f layer before the SPPF layer in the backbone network to form a C2f-CoT module. This module can improve the model's attention to small targets and closely arranged targets, reduce the cases of missed detections and false detections, thereby improving the detection accuracy of the entire model.

[0036] (3) The present invention designs a new small target detection head based on low-level, high-resolution feature maps. This detection head is designed specifically to improve the small target detection ability. By fully utilizing the information in the high-resolution feature maps, it improves the detection accuracy of the model for small targets, especially performing well when facing densely arranged targets.

[0037] (4) The present invention combines Inner-IoU and CIoU loss functions and proposes an Inner-CIoU loss function. This new loss function adjusts the auxiliary bounding box through a scaling factor, reasonably manages the generation process of auxiliary bounding boxes under different-sized targets, not only accelerates the convergence speed during training but also improves the detection accuracy for small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0039] Figure 1 is the structural diagram of the artificial cultivation shellfish larvae health status early warning model based on enhanced YOLO according to the embodiment of the present invention;

[0040] Figure 2 is the structural diagram of the Context Guided (CG) module according to the embodiment of the present invention;

[0041] Figure 3 is the structural diagram of the Contextual Transformer (CoT) module according to the embodiment of the present invention;

[0042] Figure 4Schematic diagram of the small target detection head according to an embodiment of the present invention. Detailed implementation manners

[0044] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0045] It should be noted that the steps shown in the flowchart of the drawings may be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0046] Embodiment 1

[0047] As Figure 1 shown, in this embodiment, a method for warning the health status of artificially cultivated shellfish larvae based on enhanced YOLO is provided, including:

[0048] Obtain shellfish larva images, preprocess the shellfish larva images to obtain a preprocessed data set;

[0049] Optimize the YOLOv8 model based on the CG module and the context self-attention mechanism to construct a deep learning model;

[0050] Train the deep learning model with the preprocessed data set to obtain a larva health status recognition model; wherein, the loss function of the deep learning model is the Inner-CIoU loss function;

[0051] Identify the health status of the larvae based on the larva health status recognition model.

[0052] To ensure that the annotation of the data set is both accurate and widely applicable, in this embodiment, a labeling tool is used to manually label all the images. To maintain the consistency of the annotation, the research team has developed strict standards to ensure that each bounding box can accurately reflect the target object in the image. After the labeling is completed, all the annotation information is saved in a txt file for subsequent use of the training data set to train the model and the validation and test data sets to evaluate the performance of the model. During this process, 90% of the RGB shellfish larva images are randomly divided into a training set and a validation set according to a ratio of 7:2, and the remaining images are reserved as a test set to evaluate the detection ability and robustness of the proposed model.

[0053] Training a deep neural network requires a large amount of image data support. Using data augmentation techniques can increase the number of images, thereby significantly improving the performance of the model. To increase the number of images, reduce the risk of model overfitting, and enable it to better learn the internal laws of the data, this embodiment adopts data augmentation techniques. Specifically, the dataset is expanded by performing various operations on the original images, such as brightness transformation, multi-angle rotation, and adding noise. This not only increases the number of images but also helps the model learn more feature variations, thereby improving the overall performance of the model. The data augmentation process is as Figure 2 shown.

[0054] In this embodiment, the Context Guided (CG) module is used to replace the original downsampling module of the model, helping the model better understand the relationship between shellfish larvae and their surrounding structures, integrating global and local context information, and thus improving the recognition accuracy of shellfish larvae.

[0055] The structure of the CG module is as Figure 3 shown, mainly consisting of four parts: a local feature extractor, a surrounding context extractor, a joint feature extractor, and a global context extractor. First, the local feature extractor learns local features from eight adjacent surrounding feature vectors through a standard 3×3 convolutional layer; while the surrounding context extractor uses a 3×3 dilated convolution to capture the surrounding context information with its larger receptive field. Next, the joint feature extractor fuses these two types of features and optimizes them through batch normalization and parametric ReLU activation functions. Then, the global context features are collected through a global average pooling layer, and the global information is extracted through a multi-layer perceptron. Finally, the output feature map is generated through a weighted operation.

[0056] In this embodiment, the Contextual Transformer (CoT) attention mechanism is combined with the model to enhance the ability to extract global context feature information. By introducing global context information at the end of the backbone network, it can better capture the global semantic information of the image, combine shellfish larvae with the background and other factors, and thus improve the accuracy of small target detection. The CoT module is a context self-attention module, different from traditional self-attention. It combines context information with the self-attention mechanism, makes full use of the context relationship between adjacent units, enhances the effect of self-attention learning, and improves the expressive ability of the output features.

[0057] Figure 4In it, assume that the size of the input feature map received by the CoT module is H×W×C. First, apply k×k grouped convolutions to process all adjacent K keys within the k×k grid of the input feature map to extract the static context information K1 ∈ R H×W×C. After obtaining K1, it is concatenated with the input Q key, and through two 1×1 convolutions, an attention matrix is obtained. Different from the traditional K-Q method, the new attention matrix combines the Q feature and the context K feature to enhance self-attention learning. Finally, the attention matrix is multiplied by the V value processed by 1×1 convolution to obtain the global context information K2, which is fused with the static context K1. Using the dynamic context information between adjacent keys to guide self-attention learning, the model can better understand the relationships between different positions of the feature map and avoid ignoring key feature information.

[0058] The YOLOv8 model adopts a three-layer scale feature layer design in its neck network structure. For an input image of 640×640, it uses 20×20, 40×40, and 80×80 downsampled feature maps for object detection. However, in the shellfish larvae dataset, shellfish larvae usually occupy a small area of the image and are easily occluded and overlapped. After multiple downsampling and pooling operations, most of the detailed features are lost, resulting in frequent missed detections and false detections.

[0059] In this embodiment, a small object detection head of 160×160 is integrated into the detection layer of the baseline model to improve the detection ability of small objects in the image. The new detection head structure is shown in Figure 5. First, the 80×80 scale feature map of the second layer of the backbone network is stacked up with the upsampled feature layer in the neck network. After passing through the C2f layer and upsampling, more feature layers with more small object feature information are obtained. Subsequently, these feature layers are concatenated and fused with the 160×160 feature map output by the second layer of the backbone network, enhancing the expression ability of the 160×160 scale fused feature layer for small objects. This newly added detection head enables the small object feature information to be transmitted along the downsampling path to the other three scale feature layers, so as to extract small object features in a deeper network. Through such structural optimization, the model can more accurately detect individual shellfish larvae in the image.

[0060] The design of the loss function plays a crucial role in the overall performance of the model. The CIoU loss function is the original loss function of the YOLOv8 model. Given the predicted bounding box B and the ground truth bounding box B gt , the CIoU loss is expressed as follows:

[0061]

[0062] where IoU represents the intersection of B and B gt ; c represents the diagonal length of the smallest bounding box that simultaneously contains B and B gt . ρ2 (b, b gt ) represents the Euclidean distance between B and B gt 's center points. Additionally, considering the difference in the aspect ratios of B and B gt , α and υ are adopted. α is an adjustment factor used to adjust the weight according to the value of IoU. υ is a shape difference factor that measures the similarity of shapes by comparing the angular differences in the aspect ratios of the predicted bounding box and the ground truth bounding box. The formula is as follows:

[0063]

[0064] In the formula, w and h represent the width and height of B respectively, and w gt , h gt represent the width and height of B gt respectively.

[0065] In this embodiment, the CIoU loss function is optimized into the Inner - CIoU loss function, which manages the generation of supplementary bounding boxes at different scales with a scale factor, thereby accelerating the convergence speed and improving the detection accuracy. The definition of Inner - CIoU is as follows:

[0066]

[0067]

[0068] union=(w gt *h gt )*(ratio) 2 +(w * h)*(ratio) 2 -inter;

[0069]

[0070] In the above formula, ratio is an auxiliary parameter affecting the size of the supplementary bounding box. The center points of B and B gt are represented by (x, y) and (x gt , y gt ) respectively, represent the left, right, bottom, and top boundaries of the ground truth bounding box respectively, b α , b β , b γ , b δ represent the left, right, bottom, and top boundaries of the predicted bounding box respectively, and ratio is a scale factor affecting the size of the supplementary bounding box.

[0071] This embodiment uses a self - built dataset of shellfish larvae for experiments, as shown in Table 1 - Table 2;

[0072] Table 1

[0073]

[0074] After integrating all improvement strategies, this embodiment is superior to the baseline model in terms of accuracy, recall, and mAP@0.5. The experimental results demonstrate that each improvement strategy makes a positive contribution to the performance of the method model.

[0075] (2) This embodiment also compares with models similar to current research, and the method of this embodiment has the best accuracy.

[0076] Table 2

[0077]

[0078] It can be found from the experimental results that this embodiment has the best effect.

[0079] Based on the above experimental results, the method proposed in this embodiment can effectively identify shellfish larvae and accurately classify the health status of the larvae, providing a feasible solution for the early warning task of the health status of artificially cultivated shellfish larvae.

[0080] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor. When the processor executes the computer program, the method is implemented.

[0081] On the other hand, this embodiment also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the method is implemented.

[0082] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for warning the health status of artificially cultivated shellfish larvae based on enhanced YOLO, characterized in that, Including: Obtain images of shellfish larvae, preprocess the images of shellfish larvae to obtain a preprocessed dataset; Optimize the YOLOv8 model based on the CG module and the context self-attention mechanism to construct a deep learning model; Train the deep learning model with the preprocessed dataset to obtain a larval health status recognition model; wherein, the loss function of the deep learning model is the Inner-CIoU loss function; Identify the health status of the larvae based on the larval health status recognition model.

2. The method according to claim 1, characterized in that, The process of obtaining the preprocessed dataset includes: Obtain the images of shellfish larvae, manually annotate the images of shellfish larvae to obtain annotated images; Perform brightness transformation, multi-angle rotation and noise addition operations on the annotated images in sequence to obtain an augmented dataset; Randomly divide the augmented dataset to generate the preprocessed dataset.

3. The method according to claim 1, wherein The process of constructing the deep learning model includes: Construct a CG module based on a local feature extractor, a surrounding context extractor, a joint feature extractor and a global context extractor; Construct a context self-attention mechanism module based on context information and the self-attention mechanism; Replace the original downsampling module of the YOLOv8 model with the CG module, and introduce the context self-attention mechanism module at the end of the backbone network of the YOLOv8 model to complete the construction of the deep learning model.

4. The method according to claim 3, wherein The operation process of the CG module includes: Learn local features in adjacent feature vectors through the local feature sensor; Capture surrounding context information based on the surrounding context extractor; Fuse the local features and the surrounding context information based on the joint feature extractor, optimize through batch normalization and the parametric ReLU activation function, then collect global context features through the global context extractor, extract global information through a multi-layer perceptron, and finally generate an output feature map through a weighted operation.

5. The method according to claim 3, wherein The operation process of the context self-attention mechanism module includes: Receive an input feature map, process all adjacent key vectors within the k×k grid in the input feature map based on k×k group convolutions, and obtain static context information based on the adjacent key vectors; Concatenate the static context information with the input query vector, and obtain an attention matrix through two 1×1 convolutions; Multiply the attention matrix by the value vector processed by 1×1 convolution to obtain global context information; Aggregate the static context information and the global context information to complete self-attention learning.

6. The method according to claim 1, characterized in that, The process of optimizing the YOLOv8 model based on the CG module and the context self-attention mechanism further includes: integrating a small target detection head of 160×160 in the detection layer of the YOLOv8 model; The operation process of the small target detection head includes: stacking the 80×80 scale feature map of the second layer of the backbone network with the upsampled feature layer in the neck network. After passing through the C2f layer and upsampling processing, a feature layer with small target feature information is obtained. The feature layer with small target feature information is spliced and fused with the 160×160 feature map output by the second layer of the backbone network and then output to complete the detection of small target features.

7. The method according to claim 1, characterized in that The defined expression of the Inner-CIoU loss function is: Among them, is the Inner-CIoU loss function, is the CIoU loss function, and IoU is the intersection of B and B gt , where B is the predicted bounding box, and B gt is the ground truth bounding box, and IoU inner is the inner intersection over union.

8. An electronic device, comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method described in any one of claims 1-7.

Images