Underwater fish lightweight detection method based on improved YOLOv10

Abstract

The invention provides an underwater fish lightweight detection method based on improved YOLOv10, and aims to solve the problems of feature fusion applicability, network structure calculation burden, information aggregation efficiency among different resolution feature layers and the like in fish tracking in an underwater fish culture environment. The method comprises the steps of data preprocessing, attention feature fusion, network model lightening, cross-scale multi-head decoupling detector aggregation and training verification by using YOLOv10. In model training, the prediction performance is improved by optimizing parameters and structures, comparison experiment verification is carried out, reliability is shown, and it is indicated that the method has wide application potential in the intelligent fishery industry.

Description

Technical Field

[0001] The present invention relates to the fields of computers and aquaculture Background Art

[0002] Aquaculture, as an important part of fishery, has become one of the fastest-growing technologies in global food production. With the rapid increase in the global population and the rising demand for aquatic products, the aquaculture industry is facing increasing pressure.

[0003] Fish aquaculture is the main aquatic product in China, accounting for more than 50% of the total aquaculture; followed by shellfish products and crustacean products, accounting for 25% and 14% respectively. According to the latest data, between 2019 and 2022, the scale of China's aquaculture industry increased from 976.2 billion yuan to 1.2502 trillion yuan, and the output value in 2023 exceeded 1.3 trillion yuan, with an increase of 4.49%. The demand for China's aquaculture industry will continue to grow steadily in 2024, and the demand for high-quality aquatic products by consumers will drive enterprises to continuously carry out technological innovation and model upgrading.

[0004] In recent years, due to problems such as overfishing and environmental pollution, the population of many fish species has decreased significantly, posing a severe challenge to the sustainability of aquatic resources. Therefore, accurately grasping the population quantity and distribution of fish is particularly important for the scientific management and protection of fish resources.

[0005] Traditional methods for analyzing the growth status and behavior of fish, such as manual statistics and acoustic technology, although they can provide certain monitoring effects, generally have defects such as high cost, low efficiency, and poor accuracy. Traditional fish target detection methods mainly rely on artificial feature extraction (such as color, texture, etc.) and machine learning classification models, but these methods often cannot adapt to complex underwater environments. Underwater, due to light changes, turbid water quality, and the diversity of fish appearance characteristics (such as different postures, sizes, and species), the effectiveness of traditional methods is limited. The complexity of the underwater environment may also lead to abnormal situations such as fish diseases, contaminated parasites, etc., which will affect the quantity, quality, and welfare of fish, causing significant losses to the aquaculture industry. Therefore, it is imperative to improve the yield and quality of aquaculture.

[0006] By monitoring and analyzing the growth status and behavior of fish at different growth stages, aquaculture managers can more accurately grasp the changes in fish abundance, body length, and behavior, thereby optimizing aquaculture management strategies, reducing risks caused by diseases or overfeeding, and improving the productivity and profitability of aquaculture.

[0007] With the development of agricultural modernization, intelligent fishery has gradually become an emerging trend in the fishery field. Relying on automated equipment and intelligent systems, intelligent fishery realizes real-time monitoring and management of aquaculture. In this context, fish target detection technology plays a key role. By deploying a deep learning-based target tracking algorithm on underwater camera equipment, the behavior of fish can be monitored in real time, and the health status and quantity changes of fish can be evaluated, thus providing technical support for intelligent fishery.

[0008] Image-based target detection can effectively identify and count the population quantity and distribution of different fish species, providing data support for the management of aquatic resources. The application of deep learning technology in target detection can greatly improve the accuracy and efficiency of fish target detection. Compared with traditional manual methods, the detection system based on deep learning can achieve 24-hour real-time monitoring and is not affected by human factors. This not only improves the objectivity of detection but also significantly reduces labor costs.

[0009] Underwater target tracking, as an interdisciplinary research field of computer vision and underwater intelligent technology, has important academic and practical significance. The light propagation characteristics, medium absorption and scattering effects in the underwater environment often lead to image blurring, low contrast and color distortion, seriously affecting the performance of traditional target detection and tracking algorithms. Through multi-modal data fusion, feature extraction optimization and dynamic association modeling, underwater target tracking technology can capture the multi-object interaction behavior in a dynamic environment, providing intelligent technical support for underwater robot navigation, underwater biological monitoring and underwater resource exploration. Conducting research on fish target tracking not only enriches the application of computer vision in natural scenes but also provides a new research perspective for the adaptability and robustness of vision algorithms in different environments. Summary of the Invention

[0010] The purpose of the present invention is to solve the problems existing in tracking fish in an underwater fish farming environment, namely the applicability of feature fusion, the computational burden of the network structure, and the information aggregation efficiency between feature layers of different resolutions. To solve the above three problems, the present invention proposes a lightweight underwater fish detection method based on improved YOLOv10. The concept of the present invention is to use YOLOv10 as the basic model and improve the feature extraction method by introducing lightweight convolution technology to connect multi-head detectors, so as to improve the accuracy of underwater fish detection and the target tracking ability.

[0011] According to the above invention concept, the present invention adopts the following technical solutions:

[0012] A. Data preprocessing: Label the fish body through LabelImg software to generate an XML format file. Read the XML file, convert the pixel coordinates of the target information in the file into YOLO format normalized coordinates, and write them into a TXT file.

[0013] B. Attention Feature Fusion: Use an efficient multi-scale attention mechanism (Efficient Multi-Scale Attention). By reshaping some channels into the batch dimension and grouping channels into multiple sub-features, combined with global information encoding and cross-space learning, to reduce the computational overhead while preserving channel information, thereby improving the quality of feature representation and the performance of the model.

[0014] C. Lightweight Network Model: The upsampling and downsampling layers usually use large convolutional kernels and high channel numbers, increasing the resolution of the feature map, resulting in a significant increase in the computational amount of subsequent convolutional layers and significantly improving FLOPs. Through lightweight design, the computational burden after the upsampling layer can be effectively reduced, and the overall inference speed can be improved. When connecting the detection head, a lightweight convolutional method GSConv (Group Shuffle Convolution) is used to replace the traditional SC (Standard Convolutional) convolution method, combined with the VoVGSCSP (Volumetric Grid Spatial Cross Stage Partial) module to further extract and fuse features, forming a Slim-Neck, so as to send the final features into the detection head for object detection, thereby achieving fewer parameters and lower GPU memory requirements.

[0015] D. Cross-Scale Multi-Head Decoupled Detector: Design a Detect_AFPN (Detect_Asymptotic Feature Pyramid Network) multi-head progressive feature pyramid detector. Through layer-by-layer feature aggregation, the information of low-resolution layers can be effectively transmitted to high-resolution layers, and at the same time, the detailed information of high-resolution layers is gradually fed back to low-resolution layers, so as to fully share the information between feature layers of different scales and avoid excessive calculation between feature maps.

[0016] E. Use the YOLOv10 model for training and verification: Improve the multi-scale feature extraction method and apply YOLOv10 for accurate identification of fish in the underwater environment.

[0017] While training the model, continuously adjust the model parameters to optimize its performance to ensure that the model takes into account both detection accuracy and speed in detecting underwater fish. The inventive method helps to achieve intelligent production and scientific management of precise aquaculture. And conduct comparative experiments with other series of this model. At the same time, apply the proposed model to cases such as underwater biological positioning and tracking, intelligent feeding of fish and shrimp feed, and analysis of fish feeding intensity, indicating that the present invention has a certain degree of reliability in intelligent fishery and provides a practical technical support for applying deep learning models in large-scale aquaculture scenarios. The specific process is as Figure 1 shown.

[0018] Compared with the prior art, the present invention has the following prominent substantial features and remarkable advantages: As the convolutional neural network goes deeper, the complex details of small objects gradually decrease in the feature hierarchy, posing greater challenges to detection. To solve this problem, the present invention proposes a multi-scale feature fusion strategy, which aggregates more comprehensive semantic information by fusing feature maps from different stages of the backbone network. And through the shuffle feature strategy, the combination of channel-sparse convolution and channel-dense convolution maintains the output of the original connection form with lower time complexity. The multi-head progressive feature pyramid structure proposed by the present invention supports direct interaction between non-adjacent levels, which helps to solve the problem of multi-object information conflict that may occur in the feature fusion process of different spatial positions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is an overall flowchart of the present invention.

[0020] Figure 2 is an architecture diagram based on the present invention.

[0021] Figure 3 is Figure 2 a detailed illustration of the C2F_EMA module among them.

[0022] Figure 4 is Figure 2 a detailed illustration of the GSConv module among them.

[0023] Figure 5 is Figure 2 a detailed illustration of the Detect_AFPN module among them.

[0024] Figure 6 are the experimental results of the model on the self-built dataset. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The present invention will be described in detail below with reference to the accompanying drawings.

[0026] In the present invention, the machine for running the experiment is a server cluster with A100 graphics cards, and the model is developed using the Pytorch development framework.

[0027] A. Data preprocessing.

[0028] The dataset comes from underwater salmon videos, which are manually labeled using LabelImg software, with the label set as "fish" to generate XML format files. Before model training, the XML data format is converted into TXT data format. Each line includes the category number of a target, the center coordinates of the target bounding box, the width, and the height.

[0029] B. Attention Feature Fusion.

[0030] The main principle of the Efficient Multi-scale Attention (EMA) is to reshape some channels into the batch dimension and group the channel dimension into multiple sub-features, so that the spatial semantic features are well distributed within each feature group, improving the feature representation ability. The C2F_EMA module recalibrates the channel weights in each parallel branch by encoding global information and generates better pixel-level attention for the high-level feature map. The overall structure of C2F_EMA is as Figure 3 shown.

[0031] By grouping the channel dimension, X is divided into g sub-features for learning different semantics. Without loss of generality, the learned attention weight descriptors are used to strengthen the feature representation of the regions of interest in each sub-feature. The feature grouping method enables the model to allocate and process the model on more GPU resources. This grouping method not only strengthens the feature learning of the semantic regions but also compresses the noise.

[0032] The C2F_EMA module adopts a parallel sub-network design different from traditional attention modules, using three parallel paths to extract the attention weight descriptors of the grouped feature maps. Two of the parallel paths are in the 1*1 branch, and the third path is in the 3*3 branch.

[0033] To specifically capture the dependencies across all channels and reduce the computational budget, we model the cross-channel information interaction in the channel direction. In the 1*1 branch, two 1D global average pooling operations are used to encode the channels along two spatial directions respectively, and in the 3*3 branch, after only stacking a single 3*3 kernel, the global spatial information is encoded through 2D global average pooling and the Softmax function, and the matrix dot product operation is used to aggregate the cross-space information in different spatial dimension directions. C2F_EMA can not only encode the cross-channel information to adjust the importance of different channels but also retain the precise spatial structure information in the channels.

[0034] C2F_EMA enriches the feature aggregation by providing a cross-space information aggregation method in different spatial dimension directions. Specifically, the output of the 1*1 branch encodes the global spatial information through 2D global average pooling, while the output of the 3*3 branch is directly converted to the corresponding dimension shape. Then, these outputs are aggregated through the matrix dot product operation to generate the first spatial attention map. Finally, the output feature maps within each group are aggregated through the Sigmoid function of the two generated spatial attention weight values, capturing the pixel-level pairing relationship and highlighting the global context of all pixels. The final output of C2F_EMA is the same size as the input and can be effectively stacked into modern architectures.

[0035] C. Lightweight network model.

[0036] Each spatial (width and height) compression and channel expansion of the feature map will result in partial loss of semantic information. Channel-dense convolutional computations (SC) maximize the implicit connections between each channel, but 1*1 dense convolutions consume more computing resources, while channel-sparse convolutions (DSC) completely cut off these connections. The defects of DSC are directly amplified in the backbone, whether for classification or detection. Therefore, we adopt a hybrid strategy to make the output of DSC as close as possible to SC by shuffling the features from SC and DSC-GSConv. GSConv preserves these connections as much as possible and has a lower time complexity. The structural diagram is as Figure 4 shown.

[0037] The following formulas are the FLOPs calculations for different-channel convolutions.

[0038] FLOPsSC = W * H * K1 * K2 * C1 * C2

[0039] FLOPsDSC = W * H * K1 * K2 * 1 * C2

[0040]

[0041] Among them, W is the width of the output feature map, H is the height of the output feature map, K1 * K2 is the size of the convolution kernel, and C1 and C2 are the number of channels of the input and output feature maps respectively.

[0042] The advantages of GSConv are more obvious for lightweight detectors. It benefits from the enhanced non-linear expression ability through adding DSC layers and shuffling and does not require additional FLOPs. However, if GSConv is used in all stages of the model, the network layers will be deeper, and deeper layers will exacerbate the resistance to the data flow and significantly increase the inference time. When these feature maps reach the neck, they have become slender enough (the channel size reaches the maximum, and the width and height sizes reach the minimum), and the transformation has become moderate. Therefore, a better choice is to use GSConv only at the neck.

[0043] The GS bottleneck module is based on GSConv and is used to improve the non-linear expression of features and the reuse of information, further enhancing the network's ability to process features. The VoV-GSCSP module is a cross-stage partial network module designed using a one-time aggregation method and is used for effective information fusion between feature maps at different stages. The design of these modules aims to reduce computational complexity and inference time while maintaining accuracy. Through the Slim-neck modular design, a network architecture suitable for specific tasks can be flexibly constructed according to needs.

[0044] D. Cross-scale multi-head decoupled detector.

[0045] For object tasks with scale-varying features in object detection tasks, we adopt an Adaptive Feature Pyramid Network (AFPN) to support direct interactions between non-adjacent levels. AFPN starts by fusing two adjacent low-level features and gradually incorporates higher-level features into the fusion process, which can avoid a large semantic gap between non-adjacent levels. The commonly used multi-scale feature extraction strategy is to adopt the classic top-down and bottom-up feature pyramid networks. These methods have problems of feature information loss or degradation, affecting the fusion effect between non-adjacent levels. We consider the possible multi-object information conflicts in the feature fusion process at each spatial position and further use adaptive spatial fusion operations to mitigate these inconsistencies. The architecture diagram is as Figure 5 shown.

[0046] This framework extracts the last layer of features from each feature layer of the Backbone, resulting in a set of features at different scales, denoted as {C2, C3, C4, C5}. To perform feature fusion, first, the Low-Level features C2 and C3 are input into the feature pyramid network, then C4 is added, and finally C5 is added. After the feature fusion step, a set of multi-scale features {P2, P3, P4, P5} is generated.

[0047] During the bottom-up feature extraction process of the Backbone network, Detect_AFPN progressively integrates Low-Level, High-Level, and top-level features. Specifically, Detect_AFPN initially fuses Low-Level features, then fuses deep features, and finally fuses the most High-Level features, i.e., the most abstract features. The semantic gap between non-adjacent level features is greater than that between adjacent level features, especially for bottom and top features. This directly leads to a poor fusion effect for non-adjacent level features. Therefore, it is unreasonable to directly use C2, C3, C4, and C5 for feature fusion. Since the architecture of Detect_AFPN is progressive, this will make the semantic information of different Level features closer during the progressive fusion process, thus alleviating the above problems. For example, the feature fusion between C2 and C3 reduces their semantic gap. Since C3 and C4 are adjacent level features, the semantic gap between C2 and C4 is reduced.

[0048] Adaptive Spatial Fusion (ASFF) is used to assign different spatial weights to features at different levels during the multi-level feature fusion process, enhancing the importance of key levels and reducing the impact of conflicting information from different objects. Let represent the feature vector at position (i, j) from layer n to layer l. The resulting feature vector is obtained through the adaptive spatial fusion of multi-level features and is a linear combination of the feature vectors as follows:

[0049]

[0050] where and represent the spatial weights of the four-level features at layer l and are subject to constraints.

[0051] E. Use YOLOv10 for accurate identification of fish in the underwater environment. Refer to Figure 6 They show the comparison of the present invention with other models. The experimental results show that the proposed lightweight underwater fish detection method based on improved YOLOv10 is superior to other mainstream identification methods in terms of various performances. In summary, the lightweight underwater fish detection method based on the improved YOLOv10 of the present invention establishes an efficient underwater fish detection model by improving the feature extraction method and introducing lightweight convolution technology to connect the multi-head detector. This model can extract fish features from blurred underwater environment images. By fusing the feature extraction method, it effectively reduces the influence of the non-adjacent hierarchical effect and improves the calculation speed at the same time. This research has significant academic value and broad application potential, and will make positive contributions to sustainable fishery management and ecological protection.

Claims

1. A method for underwater fish target detection, comprising the following steps: A. Perform data preprocessing on underwater farmed fish images and convert the XML file marking underwater fish bodies into normalized coordinate representation in YOLO format; B. Attention feature fusion; C. Lightweight network model; D. Aggregate cross-scale multi-head decoupled detectors; E. Use YOLOV10 for training and verification.

2. The method according to claim 1, wherein the preprocessing step comprises preprocessing the underwater fish image. The data preprocessing steps are as follows: Label the underwater fish using LabelImg software and generate an XML file. Read the XML file, convert the pixel coordinates of the target information in the file into normalized coordinates in YOLO format, and write them into a TXT file.

3. According to the method of claim 1, C2f_EMA is used to reshape some channels of the feature layer to batch dimensions, and global information encoding is combined to perform cross-spatial scale learning.

4. According to the method of claim 1, a lightweight convolution method is used to form Slim-Neck, and the final features are sent to the detection head for target detection.

5. According to the method described in claim 1, a multi-head progressive feature pyramid detector is designed to fully share information between feature layers of different scales through layer-by-layer feature aggregation.

6. According to the method of claim 1, the features of the underwater marked fish body are sent to YOLOV10 for accurate identification of fish in the underwater environment.

Images