Underwater target detection method based on improved YOLOv8 lightweight model
By introducing RFAConv, CCFM-D and EMA attention mechanisms in the YOLOv8 model, feature extraction and information fusion are improved, and the problem of poor feature extraction effect and large calculation amount in underwater target detection is solved, achieving efficient and lightweight Underwater target detection.
Patent Information
- Application Number
- CN202411146738.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2025-01-03
AI Technical Summary
The existing YOLOv8 model has poor feature extraction and information fusion effects in underwater target detection, resulting in low detection accuracy and excessive model calculation and parameters, which cannot meet the needs of fast and accurate detection.
The RFAConv module, which introduces the sensory field attention mechanism, replaces CBS and C2f, combines the CCFM-D module and EMA attention mechanism, improves the SPPF network structure, improves the ability to extract and information fusion, and reduces the amount of calculation and parameters.
It improves the accuracy and efficiency of underwater target detection, reduces resource consumption, adapts to complex and changeable underwater environments, realizes lightweight deployment, and improves detection accuracy and reduction of computing costs.
Smart Images
Figure CN119251461A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of underwater target detection, and in particular relates to an underwater target detection method based on an improved YOLOv8 lightweight model. Background Art
[0002] Underwater target detection is the process of identifying and detecting objects, animals or other underwater targets in an underwater environment by using different sensors and technologies. With the continuous development of underwater target detection, target detection methods based on deep learning have broad economic prospects in the marine field. Underwater target detection plays an important role in marine aquaculture, development of marine resources, early warning of marine disasters and protection of the marine environment. At the same time, it also faces many technical challenges. Take marine aquaculture as an example: 1) Due to the subtle changes in indicators such as seawater temperature and salinity in specific aquaculture areas, it is difficult to accurately monitor in real time. In particular, slight fluctuations in temperature have a drastic impact on the daily activities, communication and behavior of marine economic organisms. When the seawater temperature exceeds 33 degrees, sea cucumbers will die on a large scale, resulting in serious economic losses. 2) Due to factors such as tides, wind and waves, the underwater environment changes rapidly and is full of unknowns and difficulties, resulting in limited underwater operation time. This means that divers with rich underwater operation experience must complete their tasks within a specific time, and the risks encountered by divers are greatly increased. These limitations require underwater target detection models to be lightweight and efficient.
[0003] The YOLO series is a single-stage target detection algorithm. Its full name is You Only Look Once, which means that the category and location of the target can be obtained by processing the image only once. YOLOv8 was proposed in 2023. It combines the advantages of previous generations of YOLO. It is a single-stage target detection algorithm with fast detection speed and improved target detection accuracy and stability. The Backbone of YOLOv8 consists of CBS, C2f, and SPPF. The CBS module contains a convolutional layer, a batch normalization layer, and a SiLU activation function. Among them, the new C2f module continues the CSP idea, deepens the degree of fusion of gradient information, can learn more feature information and achieve lower computational complexity, which enables YOLOv8 to maintain high accuracy while improving detection speed. SPPF is a module used in YOLOv5, which can optimize the process of feature extraction and encoding and improve computing speed. Bottleneck is a special residual structure that is usually used to reduce the number of parameters and computation while maintaining network performance. The Neck part of YOLOv8 continues the PAN (path aggregation network, PAN) + FPN (feature pyramid network, FPN) idea, processes multi-scale features from Backbone, and enables the model to fuse low-level detail features with high-level semantic features through upsampling and downsampling operations, enhance the model's perception ability, and improve the accuracy of object detection at different scales. YOLOv8 adds the most core change in the Head part - decoupled-head, decomposing the original detection head into two parts, so that they focus more on their respective tasks and improve the accuracy of real-time detection.
[0004] At present, there are relevant patent documents that record the application of the traditional YOLOv8 model to underwater target detection. However, the CBS and C2f feature extraction and information fusion effects in the YOLOv8 model are poor, and the detection accuracy of underwater target information in complex scenes cannot be guaranteed. The traditional PAN+FPN network structure will lead to huge amounts of calculations and parameters. The huge amount of calculations will increase the loss of computing resources and prolong the detection time. The entire traditional YOLOv8 model has poor detection accuracy and the model is not lightweight enough, which cannot achieve the purpose of fast and accurate underwater target detection. Summary of the invention
[0005] The purpose of the present invention is to design an underwater target detection method based on an improved YOLOv8 lightweight model, which can solve the problems of poor feature extraction and information fusion effects of existing models in the background technology, and lack of lightweight and efficient effects. The present invention focuses on underwater target information and increases detection accuracy. The present invention can reduce the volume of underwater methods, reduce resource consumption, adapt to complex and changeable underwater environments, and lay the foundation for lightweight deployment of underwater equipment.
[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0007] An underwater target detection method based on an improved YOLOv8 lightweight model is characterized in that: the method is as follows:
[0008] Step 1: Obtain the public underwater target detection dataset DUO;
[0009] Step 2: Input the underwater target detection dataset DUO into the improved lightweight YOLOv8 model for training to obtain a lightweight underwater target detection model;
[0010] Step 3: Input the underwater target image to be identified into the lightweight underwater target detection model to obtain a detection result;
[0011] The lightweight YOLOv8 model includes RFAConv with receptive field attention mechanism, CCFM-D module formed by fusing CCFM and Dysample, and improved SPPF with CE module formed by fusing CBS and EMA attention mechanism.
[0012] Specifically, the improved lightweight YOLOv8 model is based on the traditional YOLOv8 model and makes the following three improvements:
[0013] In order to solve the problem of poor feature extraction and information fusion of traditional CBS and C2f, RFAConv is introduced to improve them. RFAConv multiplies the receptive field feature and the attention feature map to output, which can better utilize the spatial feature information and improve the model's expression ability, including replacing CBS and improving C2f.
[0014] Combine the cross-scale feature-fusion (CCFM) module with the dynamic upsampler Dysample. Introduce CCFM-D into Neck, which increases the model's expressiveness while achieving lightweight.
[0015] The CE module is placed in SPPF, and the EMA attention mechanism is integrated after the convolution operation based on the traditional CBS forward propagation process. The batch dimension is well reshaped to focus on small and dense underwater organisms.
[0016] Furthermore, the RFAConv of the receptive field attention mechanism replaces CBS, and the RFAConv feature extraction steps are as follows:
[0017] S1: Perform global average pooling calculation on the initial input original underwater image to generate an underwater global information feature map; perform 1×1 group convolution calculation on the underwater global information feature map to generate multiple detailed underwater feature maps; perform Softmax operation on multiple detailed underwater feature maps to generate a uniformly distributed underwater receptive field attention feature map, so as to perform the product operation in the subsequent calculation; while generating the receptive field attention feature map, perform 3×3 group convolution calculation on the initial input data feature map to generate an underwater receptive field space feature map; perform normalization operation on the receptive field space feature map to generate an underwater receptive field space feature map that meets the requirements;
[0018] S2: Performing a ReLU activation operation on the underwater receptive field spatial feature map that meets the requirements to generate the final underwater receptive field spatial feature map;
[0019] S3: Perform product operation on the generated underwater receptive field attention feature map and underwater receptive field spatial feature map, RFA underwater feature map, where the global average pooling operation adds all pixel values in each feature map and then divides it by the total number of pixels to produce a single value as the output of the feature map, and the grouped convolution divides the input feature map into multiple groups and applies the convolution operation on each group; a group of input values is processed by the activation function Softmax to produce a probability distribution.
[0020] Furthermore, the RFAConv of the receptive field attention mechanism improves C2f. Due to the problem of the decreased spatial information integration ability of BottleNeck in C2f, the BottleNeck module in C2f is improved, and the 3×3 CBS composition in the original BottleNeck is replaced with RFAConv convolution; C2f is a module proposed by YOLOv8, which has the advantages of lightweight fusion effect, and BottleNeck is a bottleneck structure in C2f, which can effectively reduce the amount of parameters and enable information to be effectively expressed. The specific steps of C2f_RFA feature fusion are as follows:
[0021] S1: Perform a Split operation on the underwater feature map generated by RFA convolution to generate two underwater feature maps of the same size;
[0022] S2: The first part flows into the BottleNeck_R module, where it is scanned by the group convolution kernel to change the size of the receptive field and generate a local RFA convolution data feature map;
[0023] S3: The second copy is used as the original underwater feature map and connected with the local RFA convolution data feature map of the first copy;
[0024] S4: Perform convolution operation on the combined feature map to generate the final underwater feature fusion feature map.
[0025] Further, the initial feature fusion matrix is subjected to the 1×1 convolution calculation; then the dimension-reduced underwater fusion information matrix is subjected to the DySample upsampling operation; then the underwater fusion information matrix after the upsampling operation is connected with the RFA underwater feature matrix to generate an underwater fusion feature matrix; finally, the underwater fusion feature matrix of the connection operation is subjected to the final feature fusion operation to generate a feature matrix suitable for detection. The CCFM module is a feature fusion network of RT-DETR, which can utilize low-level information and avoid interference from high-level semantic features. The Dysample module is a point sampling module that can avoid the fuzzy drawbacks of interpolation sampling and generate clearer feature maps.
[0026] The specific steps of CCFM-D feature fusion are as follows:
[0027] S1: Perform 1×1 convolution calculation on the initial feature fusion feature map output by the spatial pyramid pooling layer to generate a reduced-dimensional underwater fusion information feature map;
[0028] S2: Perform DySample upsampling operation on the dimension-reduced underwater fusion information feature map to facilitate subsequent detection of smaller underwater objects;
[0029] S3: Connecting the underwater fusion information feature map after the upsampling operation with the RFA underwater feature map to generate an underwater fusion feature map;
[0030] S4: Perform the final feature fusion operation on the underwater fusion feature map of the connection operation to generate a feature map suitable for detection; the spatial pyramid pooling layer extracts features from context information of different scales to generate multi-scale outputs; the feature fusion operation enhances the transmission of gradient flow information to generate a feature fusion feature map with stronger expressive power; the Dysample upsampling operation performs a Linear operation on the feature fusion feature map to generate a high-dimensional tensor projection, and the projection to the high-dimensional tensor increases the expressive power of the model; the high-dimensional tensor projection is subjected to a 0.25 times Pixel Shuffle operation to generate an offset; the offset is added to the original offset to generate a sum offset, and a new pixel value feature map is generated to capture targets of different sizes;
[0031] S5: Grid sampling operation is performed on the offset and the original feature to generate the final feature. The Linear operation mainly learns the weight feature map and the bias vector, and adjusts the optimization algorithm through back propagation and gradient descent to increase the robustness of the model; the Pixel Shuffle operation is a mapping relationship between high-resolution and low-resolution images to generate higher-resolution images; the grid sampling operation is a method of sampling a certain regular grid to generate features with uniform distribution and small computational complexity.
[0032] Furthermore, the CE module improves the SPPF steps including: starting to execute the EMA attention mechanism after 1×1 convolution dimensionality reduction, followed by BN operation to generate a batch normalized feature map; finally, performing SiLU activation operation; among them, the EMA attention mechanism reshapes the batch processing dimension well so as to focus on small and dense underwater organisms; the purpose of BN and SiLU operations is to make the features more conducive to the expression of underwater cross-information, and EMA is an attention mechanism that can retain the information on each channel and reduce computational overhead.
[0033] Furthermore, the dataset DUO contains several pictures of underwater target organisms and the corresponding labels of underwater organism species.
[0034] The following beneficial effects can be obtained through the above technical scheme: The present invention improves YOLOv8. The current underwater target detection model is limited in deployment on underwater equipment with limited resources. In order to solve the problem of excessive parameter and computational complexity of the underwater detection model, a lightweight underwater target detection model RCE-YOLO is proposed. First, in view of the poor effect of traditional CBS and C2f feature extraction and information fusion, RFAConv is introduced to replace CBS and improve C2f. Secondly, in view of the problem of excessive parameter and computational complexity, the CCFM-D module is introduced to improve the Neck layer. Finally, the CE module is used to improve SPPF so that it focuses on underwater target information and increases detection accuracy. Based on the improvement of YOLOv8, the present invention performs underwater target detection, improves detection accuracy and reduces computational cost.
[0035] The convolution based on this mechanism in the present invention is RFAConv, which aims to better utilize the spatial features in the receptive field by dynamically adjusting or enhancing the weights of the convolution kernel when processing different spatial positions. The parameters of the traditional convolution kernel are fixed during the sliding process, while RFA allows these parameters (or weights) to be dynamically adjusted according to the distribution or importance of the features in the receptive field. This mechanism can make the network pay more attention to the features that have an important impact on the task, and ignore or reduce the attention to unimportant features.
[0036] The CCFM module of the present invention is a cross-scale feature fusion module, which improves the model's ability to extract image detail information by combining feature information of different scales. Dysample is a dynamic upsampler that does not use complex dynamic convolution, but instead uses a point sampling method that does not use traditional kernels. Point sampling can ensure good detection accuracy while maintaining high efficiency, overcoming the problem that traditional Unsample is too dependent on nearby data points. The combination of the two can better reduce the amount of calculation and the amount of parameters, and improve detection accuracy.
[0037] The present invention combines the EMA attention mechanism with its unique parallel processing capability and learning ability without reducing the channel dimension, providing higher efficiency and better performance for feature extraction and advanced feature mapping. The parallel design not only helps to improve the real-time processing capability of the model, but also helps to improve the accuracy and robustness of the model in underwater tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a schematic diagram of the method steps of the present invention;
[0039] Figure 2 This is a schematic diagram of the structure of the traditional YOLOv8 model;
[0040] Figure 3 This is a schematic diagram of the structure of the improved lightweight YOLOv8 model (RCE-YOLO) of the present invention, in which the components are put together to represent the relationship between the front and back data flows of the two components;
[0041] Figure 4 It is a schematic diagram of the structure of RFAConv of the present invention;
[0042] Figure 5 This is a schematic diagram of the EMA attention mechanism structure;
[0043] Figure 6 This is a schematic diagram of the CE structure of the present invention;
[0044] Figure 7 This is a schematic diagram of the C2f_R structure of the present invention;
[0045] Figure 8 This is an example diagram of the DUO data set in Example 2;
[0046] Fig. 9 This is an example diagram of the detection results of RCE-YOLO of the present invention;
[0047] Fig.10 A heat map visualization of densely occluded scenes;
[0048] Fig.11 Heatmap visualization of dense occlusion for a blurry scene. DETAILED DESCRIPTION
[0049] The present invention will be further described below in conjunction with the accompanying drawings:
[0050] like Figure 1-11 As shown, an underwater target detection method based on an improved YOLOv8 lightweight model is characterized in that: the method is as follows:
[0051] Step 1: Obtain the public underwater target detection dataset DUO;
[0052] Step 2: Input the underwater target detection dataset DUO into the improved lightweight YOLOv8 model for training to obtain a lightweight underwater target detection model;
[0053] Step 3: Input the underwater target image to be identified into the lightweight underwater target detection model to obtain a detection result;
[0054] The lightweight YOLOv8 model includes RFAConv with receptive field attention mechanism, CCFM-D module formed by fusing CCFM and Dysample, and improved SPPF with CE module formed by fusing CBS and EMA attention mechanism.
[0055] Specifically, the improved lightweight YOLOv8 model is based on the traditional YOLOv8 model and makes the following three improvements:
[0056] In order to solve the problem of poor feature extraction and information fusion of traditional CBS and C2f, RFAConv is introduced to improve them. RFAConv multiplies the receptive field feature and the attention feature map to output, which can better utilize the spatial feature information and improve the model's expression ability, including replacing CBS and improving C2f.
[0057] Combine the cross-scale feature-fusion (CCFM) module with the dynamic upsampler Dysample. Introduce CCFM-D into Neck, which increases the model's expressiveness while achieving lightweight.
[0058] The CE module is placed in SPPF, and the EMA attention mechanism is integrated after the convolution operation based on the traditional CBS forward propagation process. The batch dimension is well reshaped to focus on small and dense underwater organisms.
[0059] The RFAConv of the receptive field attention mechanism replaces CBS. The RFAConv feature extraction steps are as follows:
[0060] S1: Perform global average pooling calculation on the initial input original underwater image to generate an underwater global information feature map; perform 1×1 group convolution calculation on the underwater global information feature map to generate multiple detailed underwater feature maps; perform Softmax operation on multiple detailed underwater feature maps to generate a uniformly distributed underwater receptive field attention feature map, so as to perform the product operation in the subsequent calculation; while generating the receptive field attention feature map, perform 3×3 group convolution calculation on the initial input data feature map to generate an underwater receptive field space feature map; perform normalization operation on the receptive field space feature map to generate an underwater receptive field space feature map that meets the requirements;
[0061] S2: Performing a ReLU activation operation on the underwater receptive field spatial feature map that meets the requirements to generate the final underwater receptive field spatial feature map;
[0062] S3: Perform product operation on the generated underwater receptive field attention feature map and underwater receptive field spatial feature map, RFA underwater feature map, where the global average pooling operation adds all pixel values in each feature map and then divides it by the total number of pixels to produce a single value as the output of the feature map, and the grouped convolution divides the input feature map into multiple groups and applies the convolution operation on each group; a group of input values is processed by the activation function Softmax to produce a probability distribution.
[0063] Furthermore, the RFAConv of the receptive field attention mechanism improves C2f. Due to the problem of the decreased spatial information integration ability of BottleNeck in C2f, the BottleNeck module in C2f is improved, and the 3×3 CBS composition in the original BottleNeck is replaced with RFAConv convolution; C2f is a module proposed by YOLOv8, which has the advantages of lightweight fusion effect, and BottleNeck is a bottleneck structure in C2f, which can effectively reduce the amount of parameters and enable information to be effectively expressed. The specific steps of C2f_RFA feature fusion are as follows:
[0064] S1: Perform a Split operation on the underwater feature map generated by RFA convolution to generate two underwater feature maps of the same size;
[0065] S2: The first part flows into the BottleNeck_R module, where it is scanned by the group convolution kernel to change the size of the receptive field and generate a local RFA convolution data feature map;
[0066] S3: The second copy is used as the original underwater feature map and connected with the local RFA convolution data feature map of the first copy;
[0067] S4: Perform convolution operation on the combined feature map to generate the final underwater feature fusion feature map.
[0068] The CCFM module is a cross-scale feature fusion module that improves the model's ability to extract image detail information by combining feature information of different scales; Dysample is a dynamic upsampler that does not use complex dynamic convolution, but instead uses point sampling that does not use traditional kernels. Point sampling can ensure good detection accuracy while maintaining high efficiency, overcoming the problem of traditional Unsample being too dependent on nearby data points. The combination of the two can better reduce the amount of calculation and parameters and improve detection accuracy.
[0069] The initial feature fusion matrix is subjected to the 1×1 convolution calculation; then the dimension-reduced underwater fusion information matrix is subjected to the DySample upsampling operation; then the underwater fusion information matrix after the upsampling operation is connected with the RFA underwater feature matrix to generate an underwater fusion feature matrix; finally, the underwater fusion feature matrix of the connection operation is subjected to the final feature fusion operation to generate a feature matrix suitable for detection. The CCFM module is the feature fusion network of RT-DETR, which can utilize low-level information and avoid the interference of high-level semantic features. The Dysample module is a point sampling module that can avoid the fuzzy drawbacks of interpolation sampling and generate a clearer feature map;
[0070] The specific steps of CCFM-D feature fusion are as follows:
[0071] S1: Perform 1×1 convolution calculation on the initial feature fusion feature map output by the spatial pyramid pooling layer to generate a reduced-dimensional underwater fusion information feature map;
[0072] S2: Perform DySample upsampling operation on the dimension-reduced underwater fusion information feature map to facilitate subsequent detection of smaller underwater objects;
[0073] S3: Connecting the underwater fusion information feature map after the upsampling operation with the RFA underwater feature map to generate an underwater fusion feature map;
[0074] S4: Perform the final feature fusion operation on the underwater fusion feature map of the connection operation to generate a feature map suitable for detection; the spatial pyramid pooling layer extracts features from context information of different scales to generate multi-scale outputs; the feature fusion operation enhances the transmission of gradient flow information to generate a feature fusion feature map with stronger expressive power; the Dysample upsampling operation performs a Linear operation on the feature fusion feature map to generate a high-dimensional tensor projection, and the projection to the high-dimensional tensor increases the expressive power of the model; the high-dimensional tensor projection is subjected to a 0.25 times Pixel Shuffle operation to generate an offset; the offset is added to the original offset to generate a sum offset, and a new pixel value feature map is generated to capture targets of different sizes;
[0075] S5: Grid sampling operation is performed on the offset and the original feature to generate the final feature. The Linear operation mainly learns the weight feature map and the bias vector, and adjusts the optimization algorithm through back propagation and gradient descent to increase the robustness of the model; the Pixel Shuffle operation is a mapping relationship between high-resolution and low-resolution images to generate higher-resolution images; the grid sampling operation is a method of sampling a certain regular grid to generate features with uniform distribution and small computational complexity.
[0076] The steps of improving SPPF by CE module include: starting to execute EMA attention mechanism after 1×1 convolution dimension reduction, followed by BN operation to generate batch normalized feature map; finally, SiLU activation operation; among them, EMA attention mechanism reshapes the batch processing dimension well so as to focus on small and dense underwater organisms; the purpose of BN and SiLU operations is to make the features more conducive to the expression of underwater cross-information, and EMA is an attention mechanism that can retain the information on each channel and reduce computational overhead.
[0077] The DUO dataset contains several images of underwater target organisms and the corresponding labels of underwater organism species.
[0078] Example 1
[0079] This embodiment discloses a lightweight underwater target detection method based on YOLOv8 improvement, see Figure 1 , including the following steps:
[0080] Step 1: Obtain the public underwater target detection dataset DUO;
[0081] Step 2: Input the underwater target detection dataset DUO into the improved lightweight YOLOv8 model for training to obtain a lightweight underwater target detection model;
[0082] The improved lightweight YOLOv8 model is a traditional YOLOv8 model (see Figure 2 ) based on the following three aspects, and obtain the lightweight RCE-YOLO model. Figure 3 :
[0083] (1) Use RFAConv for feature extraction in the YOLOv8 backbone network. RFAConv is a new module based on the receptive field attention mechanism. It can decompose the input feature map into an attention feature map and a receptive field spatial feature map, and then adjust the convolution kernel size based on the attention feature map. Finally, the product results of the two feature maps are fused together, which can effectively utilize spatial feature information to improve the performance of convolutional neural networks.
[0084] See also Figure 4 , RFAConv includes:
[0085] 1) In order to reduce the amount of calculation and increase the global information capture of the input data, the average pooling operation (AvgPool) and 1×1 group convolution (Group Conv) are used in turn to aggregate the global information and the receptive field feature information respectively. Where X represents the initial input information, AvgPool is a traditional pooling technology, the purpose is to reduce the dimension of the initial input information and reduce the amount of calculation, and group convolution is a convolution technology that groups the initial input information according to the number of channels, the purpose is to increase the expression ability of the model, the mathematical expression is shown in formula (1):
[0086] F1=GroupConv 1×1 (AvgPool(X))(1)
[0087] 2) Input the feature F1 after the operation of formula (1) into Softmax to generate the receptive field attention matrix. Softmax is a normalization technology for probability distribution, which makes the effective feature distribution more uniform and improves the stability of the model. The mathematical expression is shown in formula (2):
[0088] A rf =Softmax(F1)(2)
[0089] 3) 3×3 groups of convolutions can obtain more receptive field spatial feature information. The receptive field spatial feature map is generated after normalization (Norm) processing and ReLU activation, where normalization processing is a feature normalization technology and ReLu is an activation function. Their purpose is to avoid abnormal receptive field spatial feature information and thus affect subsequent feature output. The mathematical expression is shown in formula (3):
[0090] F rf =ReLU(Norm(g 3×3(X)))(3)
[0091] 4) Perform a product operation to generate the final receptive field spatial feature information, where A rf and F rf The mathematical expressions representing the receptive field attention matrix and the receptive field spatial feature map are shown in formula (4):
[0092] F RFAConv =A rf ×F rf (4)
[0093] 5) Improve C2f (see Figure 7 ) performs a 1×1 convolution operation on the receptive field spatial feature information output by the previous RFAConv layer to further reduce the computational complexity and improve the expression ability of the local feature information of the receptive field space, where F RFAConv represents the spatial feature information of the receptive field output by the previous layer of RFAConv. The mathematical expression is shown in formula (5):
[0094] F'=(Conv 1×1 (F RFAConv ))(5)
[0095] 6) Perform the above RFAConv operation on the local receptive field spatial feature information to further enhance the aggregation ability of underwater target information and receptive field feature information. Where F' represents the local feature map after 1×1 convolution operation, F c2f represents the output of the improved C2f, and the mathematical expression is shown in formula (6):
[0096] F c2f =RFAConv(F')(6)
[0097] (2) In YOLOv8, the PAN+FPN network architecture helps the model improve the accuracy of multi-scale target detection, but the PAN+FPN structure is affected by the number of feature maps and the resolution size, resulting in excessive parameters and calculations. Unsample relies on surrounding pixels for sampling, which has certain limitations. To solve the above problems, the CCFM module of RT-DETR is combined with the upsampling module Dysample, and the CCFM-D module (see Figure 4 )By combining feature information of different scales, the model's ability to extract image detail information is improved. Dysample overcomes the problem that the Unsample interpolation result deviates from the true value through point sampling.
[0098] 1) PAN+FPN represents a multi-scale feature network structure, Unsample is a traditional upsampling technique that relies on the similarity values of image neighbors for sampling. RT-DETR is another end-to-end object detection model, the CCFM module is the feature fusion network inside RT-DETR, and Dysample is a new type of point sampler that relies on point sampling to collect effective information.
[0099] 2) Dimensionality reduction through 1×1 convolution can further retain underwater key information and reduce redundant calculations. Linear operation is performed on the 1×1 convolution feature. CE Representatives from SPPF-CE (see Figure 6 ) output features, Linear is a linear transformation operation that produces a high-dimensional tensor projection, which increases the expressiveness of the model. The mathematical expression is shown in formula (7):
[0100] F2=Linear(Conv 1×1 (F CE ))(7)
[0101] 3) Perform a 0.25 times pixel shuffle operation on the high-dimensional tensor projection to generate an offset. Add the offset to the original feature to generate a sampling set. The pixel shuffle operation can maintain the information integrity of the underwater feature map to the greatest extent and reduce the blur or distortion that may be introduced during the upsampling process. O represents the offset after 0.25 times pixel shuffle, F2 represents the feature after linear transformation, and the mathematical expression is shown in equations (8)-(9):
[0102] O=0.25(Pixel Shuffle(F2))(8)
[0103] S=O+F CE (9)
[0104] 4) Perform grid sampling (grid_sampling) on the sample set and the original input features to generate the final features. The purpose of grid sampling is to learn complex spatial transformation relationships and increase the rich expression of underwater sampling information. S represents the sum of the original features and the offset after pixel shuffling. The mathematical expression is shown in formula (10):
[0105] F grid =grid_sample(F CE ,S)(10)
[0106] 4) Perform 1×1 convolution dimensionality reduction on the output features of the C2f module from the feature extraction layer. Concat the dimensionality reduction features with the Dysample output features.b Represents the features from the backbone network, C2f b represents the C2f operation from the backbone network, F grid represents the feature of the grid sampling output, Concat represents the connection operation, which is used to connect the same size. The mathematical expression is shown in formula (11):
[0107] F3=Concat(F grid ,Conv 1×1 (C2f b (F b )))(11)
[0108] 5) The Concat features are sent to the C2f of the feature fusion layer for further fusion processing, where F3 represents the features output by Concat, and C2f Neck The C2f module represents the feature fusion layer, F final The mathematical expression representing the final generated fusion feature is shown in formula (12):
[0109] F final =C2f Neck (F3)(12)
[0110] (3) Design the CE module (Conv-EMA, CE), see Figure 6 , increase the retention of key underwater information and further improve the detection accuracy. CE modules (such as Figure 6 As shown in Figure 2, the EMA attention mechanism (as shown in Figure 2) is added to the traditional CBS forward propagation process. Figure 5 After being fused to the convolution operation, the batch dimension is reshaped nicely to focus on small and dense underwater creatures.
[0111] 1) After the 1×1 convolution dimension reduction, the EMA attention mechanism is started. In order to learn more interactive information between channels, the original feature F k Perform group convolution operation. Divide it into 3 parts: F h 、F w 、F c , and then group the features F h 、F w Perform global average pooling to further obtain global information. Feature F c The initial underwater feature information can be well preserved to avoid information loss. Among them, GAP represents the global average pooling operation, Group represents the group convolution operation, and performing group convolution before global average pooling can further reduce the number of parameters and calculations, and extract rich underwater global features. F represents the module output feature before the SPPF-CE module, as shown in equations (13)-(14):
[0112] F k =Conv 1×1 (F)(13)
[0113] F h =GAP(Group(F k ))(14)
[0114] F w =GAP(Group(F k ))(15)
[0115] F c =Conv 3×3 (Group(F k ))(16)
[0116] 2) Grouping feature F h 、F w Perform the Concat operation and the 1×1 convolution operation in sequence, and then split the features (Split), sigmoid and multiply them to obtain the feature F s , as shown in equations (17)-(18): Among them, Sigmoid is an activation function, the purpose of which is to avoid overfitting of the model. The role of Concat is to connect the feature information of the two branches, thereby enriching the expressive power of the features. Then the key information is locked through 1×1 convolution. The purpose of the final split operation is to obtain richer underwater gradient information.
[0117] X h ,X W =Split(Conv 1×1 (Concat(F h ,F w )))(17)
[0118] F s =Sigmoid(X h )*Sigmoid(X W )(18)
[0119] 3) First, simultaneously calculate the F of formula (18) s and segmentation feature F w Perform group normalization (GN) operation, average pooling (AP), Softmax operation, Matmul and average pooling, Softmax operation and Matmul operation to obtain F ms 、F mc As shown in equations (19)-(20): ms and F mcPerform the add operation to output the cross-space feature. The cross-feature output is shown in formula (21): Where AP is the average pooling operation. BN is batch normalization (Batch Norm, BN). The BN operation will bring a lot of calculation. Using the group normalization (Group Norm, GN) operation can reduce a certain amount of calculation. The Softmax operation is beneficial to the subsequent underwater cross feature F cross Fusion. The Matmul operation is to quickly match the dimensional information from different scales, which is conducive to the expression of underwater cross information. Add represents the “+” operation of formula (21). c It is the feature after 3×3 convolution.
[0120] F ms =Matmul(Softmax(AP(GN(F s ))),F c )(19)
[0121] F mc =Matmul(Softmax(AP(F c )),GN(F s ))(20)
[0122] F cross =F ms +F mc (twenty one)
[0123] 4) Perform Sigmoid operation on the underwater cross features, adjust the weights of the cross spatial features and the initial features that have undergone group convolution, and generate the final module output F EMA , as shown in formula (22):
[0124] F EMA =Sigmoid(F cross )*Groups(F)(22)
[0125] 5) The features output by EMA are subjected to BN and SiLU operations in sequence, where BN is batch normalization, which is used to normalize the features. SiLU is an activation function that can avoid gradient explosion. CE The final output of the CE module is shown in formula (23):
[0126] F CE =SiLU(BN(F EMA ))(twenty three)
[0127] Step 3: Input the underwater target image to be identified into the lightweight underwater target detection model to obtain the detection result.
[0128] Example 2
[0129] This embodiment performs target detection on underwater images through specific experiments to verify the effectiveness of the method of the present invention.
[0130] Experimental platform and experimental parameters: The hardware environment of this embodiment uses a 13th Gen lntel (R) Core i5-13490F processor, an RTX 4060Ti graphics card, 16GB video memory, a Windows 10 operating system, a Pytorch development environment, 200 training rounds, and early stopping during training to avoid overfitting. The input image is 640*640, the training momentum is set to 0.837, the iteration batch is 4, and the SGD optimizer is used to optimize the model. The initial learning rate is 0.01 and the weight decay is set to 0.0005.
[0131] Dataset: This paper uses the DUO dataset. The underwater target detection dataset DUO contains the integration of the National Underwater Robot Professional Competition Challenge (URPC) data sets for many years. The dataset mainly contains four types of marine organisms: sea cucumbers, sea urchins, starfish, and scallops. Figure 8 Schematic diagram selected for the DUO dataset.
[0132] Experimental results: Fig.10 and 11 As shown in the figure, this experiment compares the effect of incorporating this improvement into the YOLOv8 algorithm, and at the same time compares the results with other mainstream models, involving four indicators: parameter quantity, GFLOPs, mAP50 and mAP50:90, and compares the model performance and complexity. Through comparison, it is found that the RCE-YOLO model proposed in this invention improves the detection accuracy while reducing the parameter quantity and GFLOPs by 32.3% and 0.9G respectively, and the mAP50 and mAP50:90 values reach 83.6% and 64.2% respectively, which are 1.4% and 1.2% higher than the baseline model YOLOv8. See Table 1 for the final test results of the experiment. Fig. 9 .
[0133] Table 1 Comparison of RCE-YOLO and other mainstream models
[0134]
[0135] Table 2 RCE-YOLO ablation experiment
[0136]
[0137]
[0138] The above are all preferred embodiments of the present invention. For ordinary technicians in this technical field, without departing from the principle of the present invention, various equivalent modifications to the present invention belong to the protection scope of the claims attached to this application.
Claims
1. An underwater target detection method based on an improved YOLOv8 lightweight model, characterized in that: The method is as follows: Step 1: Obtain the public underwater target detection dataset DUO; Step 2: Input the underwater target detection dataset DUO into the improved lightweight YOLOv8 model for training to obtain a lightweight underwater target detection model; Step 3: Input the underwater target image to be identified into the lightweight underwater target detection model to obtain a detection result; The lightweight YOLOv8 model includes RFAConv of the receptive field attention mechanism, a CCFM-D module formed by fusing CCFM and Dysample, and an improved SPPF of the CE module formed by fusing CBS and EMA attention mechanisms.
2. The underwater target detection method based on the improved YOLOv8 lightweight model according to claim 1, characterized in that: The RFAConv of the receptive field attention mechanism replaces CBS. The RFAConv feature extraction steps are as follows: S1: Perform global average pooling calculation on the initial input original underwater image to generate an underwater global information feature map; perform 1×1 group convolution calculation on the underwater global information feature map to generate multiple detailed underwater feature maps; perform Softmax operation on multiple detailed underwater feature maps to generate a uniformly distributed underwater receptive field attention feature map, so as to perform the product operation in the subsequent calculation; while generating the receptive field attention feature map, perform 3×3 group convolution calculation on the initial input data feature map to generate an underwater receptive field space feature map; perform normalization operation on the receptive field space feature map to generate an underwater receptive field space feature map that meets the requirements; S2: Performing a ReLU activation operation on the underwater receptive field spatial feature map that meets the requirements to generate the final underwater receptive field spatial feature map; S3: Perform product operation on the generated underwater receptive field attention feature map and underwater receptive field spatial feature map, RFA underwater feature map, where the global average pooling operation adds all pixel values in each feature map and then divides it by the total number of pixels to produce a single value as the output of the feature map, and the grouped convolution divides the input feature map into multiple groups and applies the convolution operation on each group; a group of input values is processed by the activation function Softmax to produce a probability distribution.
3. The underwater target detection method based on the improved YOLOv8 lightweight model according to claim 1, characterized in that: RFAConv with receptive field attention mechanism improves C2f, C2f_RFA feature fusion steps: S1: Perform a Split operation on the underwater feature map generated by RFA convolution to generate two underwater feature maps of the same size; S2: The first part flows into the BottleNeck_R module, where it is scanned by the group convolution kernel to change the size of the receptive field and generate a local RFA convolution data feature map; S3: The second copy is used as the original underwater feature map and connected with the local RFA convolution data feature map of the first copy; S4: Perform convolution operation on the combined feature map to generate the final underwater feature fusion feature map.
4. The underwater target detection method based on the improved YOLOv8 lightweight model according to claim 1, characterized in that: CCFM-D fusion feature steps: S1: 1×1 convolution calculation is performed on the initial feature fusion feature map output by the spatial pyramid pooling layer to generate a reduced-dimensional underwater fusion information feature map; S2: Perform DySample upsampling operation on the dimension-reduced underwater fusion information feature map to facilitate subsequent detection of smaller underwater objects; S3: Connecting the underwater fusion information feature map after the upsampling operation with the RFA underwater feature map to generate an underwater fusion feature map; S4: Perform the final feature fusion operation on the underwater fusion feature map of the connection operation to generate a feature map suitable for detection; the spatial pyramid pooling layer extracts features from context information of different scales to generate multi-scale outputs; the feature fusion operation enhances the transmission of gradient flow information to generate a feature fusion feature map with stronger expressive power; the Dysample upsampling operation performs a Linear operation on the feature fusion feature map to generate a high-dimensional tensor projection, and the projection to the high-dimensional tensor increases the expressive power of the model; the high-dimensional tensor projection is subjected to a 0.25 times Pixel Shuffle operation to generate an offset; the offset is added to the original offset to generate a sum offset, and a new pixel value feature map is generated to capture targets of different sizes; S5: Grid sampling operation is performed on the offset and the original feature to generate the final feature. The Linear operation mainly learns the weight feature map and the bias vector, and adjusts the optimization algorithm through back propagation and gradient descent to increase the robustness of the model; the Pixel Shuffle operation is a mapping relationship between high-resolution and low-resolution images to generate higher-resolution images; the grid sampling operation is a method of sampling a certain regular grid to generate features with uniform distribution and small computational complexity.
5. The underwater target detection method based on the improved YOLOv8 lightweight model according to claim 1, characterized in that: The steps of improving SPPF by CE module include: starting to execute EMA attention mechanism after 1×1 convolution dimensionality reduction, followed by BN operation to generate batch normalized feature map; finally, SiLU activation operation; among them, EMA attention mechanism reshapes the batch processing dimension well so as to focus on small and dense underwater organisms; the purpose of BN and SiLU operations is to make the features more conducive to the expression of underwater cross-information.
6. The underwater target detection method based on the improved YOLOv8 lightweight model according to claim 1, characterized in that: The DUO dataset contains several images of underwater target organisms and the corresponding labels of underwater organism species.