Underwater image target detection method and device based on deep learning, and medium

Abstract

The invention discloses an underwater image target detection method and device based on deep learning, and a medium, and aims to solve the problem of insufficient underwater optical image target recognition precision. The method comprises the steps that after an underwater optical image is collected, noise processing is carried out through self-adaptive mixed median filtering, and the filtering weight is dynamically adjusted based on a local gradient module value; the image quality is enhanced by adopting a Retinex method; a YOLOv5 model is improved, a CBAM attention mechanism module is introduced, a feature map is weighted through space and channel attention scores, a bidirectional feature pyramid network is adopted to replace a traditional FPN + PAN structure, and multi-scale feature fusion optimization is achieved. According to the bidirectional feature fusion, feature expression is enhanced through weight normalization and sampling operation, and nonlinear enhancement is carried out in combination with an optical feature map. According to the invention, through joint optimization of the preprocessing and the deep learning model, the target detection precision in the complex underwater environment is significantly improved.

Description

Technical Field

[0001] The present invention relates to a method, device and medium for underwater image target detection based on deep learning, and belongs to the technical field of underwater target recognition. Background Art

[0002] The ocean is an area that has not yet been developed on a large scale, especially the complex environment of the seabed and the rich and diverse natural resources in the deep sea. These factors have prompted countries around the world to invest a lot of manpower, material resources and financial resources in ocean exploration, especially deep-sea exploration and deep-sea operations research.

[0003] The starting point of the underwater target detection task is mainly based on underwater target images. Optical images have high resolution, realistic target depiction, and more detailed features, making them an important means of underwater target detection. However, light is severely scattered and absorbed when propagating underwater, resulting in severe quality issues in underwater optical images such as color distortion, low contrast, blurred details, and uneven illumination. In addition, the underwater environment is complex and changeable. Water turbidity, suspended matter, and lighting conditions can significantly affect image quality, making the target detection task of underwater optical images extremely challenging. Therefore, it is necessary to design a method based on underwater optical images and apply deep learning algorithms to underwater target detection tasks to solve the above technical problems. Summary of the Invention

[0004] The problem to be solved by the present invention is to improve the target recognition accuracy of underwater optical images, and propose an underwater image target detection method, equipment and medium based on deep learning.

[0005] The technical solution of the present invention:

[0006] The underwater image target detection method based on deep learning includes the following steps:

[0007] S1. Acquire underwater optical images;

[0008] S2. Denoising the underwater optical image using an adaptive hybrid median filtering method;

[0009] S3. Enhance the denoised image using the Retinex method.

[0010] S4. Build an improved YOLOv5 object detection model:

[0011] S41: Add CBAM attention mechanism module or adopt attention mechanism in YOLOv5 model;

[0012] S42: Use a bidirectional feature pyramid network to replace the original FPN+PAN structure;

[0013] S5. Input the enhanced image into the improved YOLOv5 model for target detection.

[0014] Furthermore, the adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on local gradient modulus values, which realizes dynamic filtering of noise-sensitive areas.

[0015] Furthermore, the calculation formula of the local gradient modulus is:

[0016]

[0017] in, is the local gradient modulus, is the improved Sobel horizontal gradient operator, is the vertical gradient operator, is the horizontal adjustment coefficient, is the vertical adjustment coefficient, is the cross-term compensation factor;

[0018] The calculation formula of the adaptive weight function is:

[0019]

[0020] in, is the adaptive weight function, is the regulating factor, is the hyperbolic tangent function, and k is the curvature parameter:

[0021] The expression for calculating the improved median is:

[0022]

[0023] in, , To sort the pixel values ​​in the filter window in ascending order and take the median value, is the adaptive weight, is the pixel value preprocessed by bilateral filtering, is the domain feature difference, is the nonlinear attenuation coefficient. The superscripts i and n in the summation symbol represent the summation from 1 to n.

[0024] Furthermore, step S2 also includes grayscale pre-processing before execution:

[0025] The pixel values ​​of the three channels R, G, and B are multiplied with different weights and summed to obtain the grayscale value. The calculation formula is:

[0026]

[0027] in Indicates the current pixel coordinates, Represents the pixel value after grayscale processing, Represents the red component value, Represents the green component value, Expressed as the blue component value.

[0028] Furthermore, the Retinex method in step S3 is a single-scale Retinex method.

[0029] Furthermore, a CBAM attention mechanism module is added to the YOLOv5 model: the global average pooling and standard deviation pooling operations are used on the original feature map to calculate the attention maps of the spatial feature map and the channel feature map respectively, and then the spatial attention score and the channel attention score are obtained through the sigmoid function. Finally, the obtained spatial attention score and channel attention score are multiplied with the original feature map to achieve feature map weighting;

[0030] Or use the attention mechanism:

[0031] The calculation formula is as follows:

[0032]

[0033] Among them, Query: query vector, Source: input sequence, including elements. : Input sequence length (i.e. ||Source||, indicating the dimension or number of elements of the sequence), : key vector for the i-th input element, : the value vector of the i-th input element, : Similarity between query vector and key vector;

[0034] First, the correlation between the query and different keys is calculated, that is, the weight coefficients of different values ​​are calculated; then the output of the previous stage is normalized to map the range of values ​​between 0 and 1; finally, the values ​​are weighted and summed according to the weight coefficients to obtain the final attention value.

[0035] Furthermore, the formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is:

[0036]

[0037] in, , are the first normalized fusion weight and the second normalized fusion weight respectively, is the upsampling operation, is the downsampling operation, is the output of the previous layer, is the output of the next layer, is the input feature map of the current layer (i);

[0038] The formula of the feature enhancement function is:

[0039]

[0040] in, is an optical feature map related to target saliency extracted from the input image, such as brightness channel, specific color channel, edge intensity map, etc. Tanh is the hyperbolic tangent function. is the feature enhancement function;

[0041] The formula for multi-scale output fusion is:

[0042]

[0043] in, is the gradient sensitivity coefficient, For multi-scale output fusion, is the feature map gradient.

[0044] A device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a method for underwater image target detection based on deep learning.

[0045] A medium stores a computer program, which, when executed by a processor, implements an underwater image target detection method based on deep learning.

[0046] Beneficial effects of the present invention:

[0047] 1. This paper uses a novel adaptive hybrid median filtering algorithm for image denoising. Compared to other filtering methods, this algorithm performs better in key metrics such as peak signal-to-noise ratio, mean square error, and structural similarity, effectively removing noise while better preserving image details.

[0048] 2. By applying the Retinex algorithm for image enhancement, this invention effectively mitigates or eliminates image issues (such as color distortion and low contrast) caused by uneven and scattered underwater lighting. This significantly improves overall image visibility, making details in both dark and bright areas more clearly visible.

[0049] 3. This invention improves the BiFPN structure. Residual connections are added to the model to enhance feature expression capabilities; single-input edge nodes are removed, significantly reducing the number of model parameters and accelerating model inference; and weights are added to features at each scale during the fusion process, adjusting the contribution of each scale to achieve weighted fusion of different scales.

[0050] 4. This paper adds a CBAM attention mechanism module, which employs two levels of attention: channel attention and spatial attention, capturing the interdependencies between channels and the strong correlations in spatial dimensions, respectively. Therefore, the CBAM attention mechanism enables the network to focus on important channels and spatial locations in feature representation, effectively improving feature quality and expressiveness. Dynamic sparse gating is introduced on top of CBAM to reduce computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a method for underwater image target detection based on deep learning described in the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. That is, the specific embodiments described are only part of the embodiments of the present invention, not all of the specific embodiments.

[0053] Example 1:

[0054] like Figure 1 As shown, a method for underwater image target detection based on deep learning includes the following steps:

[0055] S1. Acquire underwater optical images;

[0056] S2. Denoising the underwater optical image using an adaptive hybrid median filtering method;

[0057] Furthermore, the adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on the local gradient modulus to achieve dynamic filtering of noise-sensitive areas; the calculation formula of the local gradient modulus is:

[0058]

[0059] in, is the local gradient modulus, is the improved Sobel horizontal gradient operator, is the vertical gradient operator, is the horizontal adjustment coefficient, is the vertical adjustment coefficient, is the cross-term compensation factor;

[0060] The calculation formula of the adaptive weight function is:

[0061]

[0062] in, is the adaptive weight function, is the regulating factor, is the hyperbolic tangent function;

[0063] The expression for calculating the improved median is:

[0064]

[0065] in, , To sort the pixel values ​​in the filter window in ascending order and take the median value, is the adaptive weight, is the pixel value preprocessed by bilateral filtering, is the domain feature difference, is the nonlinear attenuation coefficient. The superscripts i and n in the summation symbol represent the summation from 1 to n.

[0066] Furthermore, the median value of the pixel value within the filter window is combined with the left and right adjacent pixel values ​​to form a combination and normalized. The normalized results are used as the weights of the three pixel values, multiplied and summed, and the final sum is used to replace the pixel value at the center of the current filter window. The process is completed; the edge-preserving denoising effect is achieved through weighted median sorting. A cross-term compensation factor γ is introduced in the gradient modulus calculation to enhance the oblique edge response characteristics. The weight function adopts the hyperbolic tangent nonlinear transformation, and the curvature parameter k is used to achieve domain adaptability. The median calculation integrates weighted sorting and nonlinear attenuation terms to optimize the denoising effect of texture areas while preserving edges.

[0067] S3. Enhance the denoised image using the Retinex method.

[0068] Furthermore, the Retinex method in step S3 is a single-scale Retinex method.

[0069] Furthermore, image enhancement, when calculating the grayscale value of a target pixel in an image, is obtained by weighting the pixel values ​​in the area centered on the target point, and the weight ratio is determined by the surround function;

[0070] The steps of the single-scale Retinex method are:

[0071] Step 1: De-noise the side-scan underwater image obtained in step S2 , perform data type conversion, changing the image from integer to double;

[0072] Step 2: Determine the smoothness c of the scale parameter Gaussian filter and determine the weight coefficient The recommended range of c is 15~200. Can be set to 1;

[0073] Step 3: Obtain value;

[0074]

[0075] : Gaussian filter (kernel size is determined by c) used to estimate the illumination component. : Minimum value (such as ), to avoid taking the logarithm about zero.

[0076] Step 4: Transformation from logarithmic domain to real number domain ;

[0077]

[0078] Step 5: Perform linear correction processing, and the image enhancement result is obtained after correction.

[0079]

[0080] S4. Build an improved YOLOv5 object detection model:

[0081] S41: Add CBAM attention mechanism module to YOLOv5 model;

[0082] S42: Use a bidirectional feature pyramid network to replace the original FPN+PAN structure;

[0083] Furthermore, Python programming language and NumPy library are used to implement time series modeling;

[0084] Furthermore, the implementation of the CBAM attention module in step S41 includes: using global average pooling and standard deviation pooling operations on the original feature map to respectively calculate the attention maps of the spatial feature map and the channel feature map, and then obtaining the spatial attention score and the channel attention score through the sigmoid function. Finally, the obtained spatial attention score and channel attention score are multiplied by the original feature map to achieve feature map weighting. The original feature map contains multi-scale features extracted by convolution, such as edges, textures, and semantic information. The channel attention score is the weight value of each channel (0~1). The spatial attention score is the weight heat map (0~1) of each pixel position;

[0085] Furthermore, dynamic sparse gating is introduced on the basis of CBAM to reduce the computational complexity. The formula of dynamic sparse attention mechanism is as follows:

[0086] The formula for channel attention enhancement is:

[0087]

[0088]

[0089] Among them, H: feature map height (Height), W: feature map width (Width), F: input feature map (Featuremap) :The feature map is located in space The eigenvalue at : global average value of feature map, , MLP: Multi-Layer Perceptron, : Sigmoid activation function; captures feature motion information by increasing standard deviation pooling.

[0090] The formula for spatial attention improvement is:

[0091]

[0092]

[0093] Among them; F: input feature map, dimension is C×H×W (number of channels×height×width), : The maximum pooling result of the feature map in the spatial dimension (H×W), dimension C×1×1, : The average pooling result of the feature map in the spatial dimension, dimension C×1×1, [;]: Splicing operation along the channel dimension (if the original number of channels is C, the number after splicing is 2C). : Convolutional layer containing a 7×7 convolution kernel to fuse spatial information. : Sigmoid activation function, mapping weights to the [0,1] interval. : Convolution operation on the feature map F. : A learnable sparsification gating parameter used to adjust the sparsity of weights at each spatial location. ⊙: Element-wise multiplication (Hadamard product).

[0094] Furthermore, the formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is:

[0095]

[0096] in, , are the first normalized fusion weight and the second normalized fusion weight respectively, is the upsampling operation, is the downsampling operation, is the output of the previous layer, is the output of the next layer, is the input feature map of the current layer (i);

[0097] The formula of the feature enhancement function is:

[0098]

[0099] in, is the optical feature map related to the target saliency extracted from the input image (such as brightness channel, specific color channel, edge intensity map, etc.), tanh is the hyperbolic tangent function, is the feature enhancement function;

[0100] The formula for multi-scale output fusion is:

[0101]

[0102] in, is the gradient sensitivity coefficient, For multi-scale output fusion, is the feature map gradient.

[0103] Furthermore, the underwater image target detection model based on deep learning can more accurately identify underwater image features, identify different types of underwater images through features, and provide more accurate image information for underwater equipment.

[0104] S5. Input the side-scan underwater image obtained in step S3 after image enhancement processing into the deep learning-based underwater image target detection model obtained in step S4 to perform deep learning-based underwater image target detection.

[0105] Example 2:

[0106] The difference between this embodiment and embodiment 1 is that, before step S2 is executed, this embodiment also includes grayscale preprocessing, multiplying the pixel values ​​of the three channels R, G, and B with different weights and summing them to obtain the grayscale value. The calculation formula is:

[0107]

[0108] in Indicates the current pixel coordinates, Represents the pixel value after grayscale processing, Represents the red component value, Represents the green component value, Expressed as the blue component value.

[0109] Place the current 3×3 filter window in the upper left corner of the original image; determine the pixel values ​​within the current filter window in order from left to right and from top to bottom. The working principle of median filtering is to sort all pixels in the local pixel composition area of ​​a given image according to the size of the pixel value, and then take the pixel in the middle position of the sort as the new pixel value in the area;

[0110] Step S41: Add the CBAM attention mechanism module to the YOLOv5 model and replace it with the attention mechanism:

[0111] The calculation formula is as follows:

[0112]

[0113] Among them, Query: query vector, Source: input sequence, including elements. : Input sequence length (i.e. ||Source||, indicating the dimension or number of elements in the sequence). : key vector for the ith input element. : Vector of values ​​for the ith input element. : The similarity between the query vector and the key vector. First, the correlation between the query and different keys is calculated, that is, the weight coefficients of different values ​​are calculated; then the output of the previous stage is normalized, mapping the range of values ​​to between 0 and 1; finally, the values ​​are weighted and summed according to the weight coefficients to obtain the final attention value.

[0114] A device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the underwater image target detection method based on deep learning as described in Examples 1 and 2.

[0115] The processor may be a general-purpose or specialized computing chip such as a CPU, GPU, DSP, ASIC, or FPGA. The memory may include, but is not limited to, storage media such as RAM, ROM, flash memory, hard disk, or optical disk.

[0116] A medium stores a computer program, which, when executed by a processor, implements the underwater image target detection method based on deep learning as described in Examples 1 and 2.

[0117] The storage medium includes non-volatile storage carriers such as solid-state drives (SSDs), USB flash drives, SD cards, optical disks, or cloud storage.

Claims

1. A method for underwater image target detection based on deep learning, characterized in that , including the following steps: S1. Acquire underwater optical images; S2. Denoising the underwater optical image using an adaptive hybrid median filtering method; S3. Enhance the denoised image using the Retinex method. S4. Build an improved YOLOv5 object detection model: S41: Add CBAM attention mechanism module or adopt attention mechanism in YOLOv5 model; S42: Use a bidirectional feature pyramid network to replace the original FPN+PAN structure; S5. Input the enhanced image into the improved YOLOv5 model for target detection.

2. The underwater image target detection method based on deep learning according to claim 1, characterized in that: The adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on local gradient modulus values, which realizes dynamic filtering of noise-sensitive areas.

3. The underwater image target detection method based on deep learning according to claim 2, characterized in that: The calculation formula of the local gradient modulus is: ； in, is the local gradient modulus, is the improved Sobel horizontal gradient operator, is the vertical gradient operator, is the horizontal adjustment coefficient, is the vertical adjustment coefficient, is the cross-term compensation factor; The calculation formula of the adaptive weight function is: ； in, is the adaptive weight function, is the regulating factor, is the hyperbolic tangent function, and k is the curvature parameter: The expression for calculating the improved median is: ； in, , To sort the pixel values ​​in the filter window in ascending order and take the median value, is the adaptive weight, is the pixel value preprocessed by bilateral filtering, is the domain feature difference, is the nonlinear attenuation coefficient. The superscripts i and n in the summation symbol represent the summation from 1 to n.

4. The underwater image target detection method based on deep learning according to claim 2, characterized in that: Before step S2 is executed, grayscale preprocessing is also included: The pixel values ​​of the three channels R, G, and B are multiplied with different weights and summed to obtain the grayscale value. The calculation formula is: ； in Indicates the current pixel coordinates, Represents the pixel value after grayscale processing, Represents the red component value, Represents the green component value, Expressed as the blue component value.

5. The underwater image target detection method based on deep learning according to claim 1 is characterized in that ,The Retinex method in step S3 is a single-scale Retinex method.

6. The underwater image target detection method based on deep learning according to claim 1 is characterized in that , Add the CBAM attention mechanism module based on the YOLOv5 model: use global average pooling and standard deviation pooling operations on the original feature map to calculate the attention maps of the spatial feature map and channel feature map respectively, then use the sigmoid function to obtain the spatial attention score and channel attention score. Finally, the obtained spatial attention score and channel attention score are multiplied with the original feature map to achieve feature map weighting; Or use the attention mechanism: The calculation formula is as follows: ； Among them, Query: query vector, Source: input sequence, including elements, : Input sequence length, i.e. ||Source||, represents the dimension or number of elements of the sequence, : key vector for the i-th input element, : the value vector of the i-th input element, : Similarity between query vector and key vector; First, the correlation between the query and different keys is calculated, that is, the weight coefficients of different values ​​are calculated; then the output of the previous stage is normalized to map the range of values ​​between 0 and 1; finally, the values ​​are weighted and summed according to the weight coefficients to obtain the final attention value.

7. The underwater image target detection method based on deep learning according to claim 5, characterized in that: The formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is: ； in, , are the first normalized fusion weight and the second normalized fusion weight respectively, is the upsampling operation, is the downsampling operation, is the output of the previous layer, is the output of the next layer, is the input feature map of the current layer (i); The formula of the feature enhancement function is: ； in, is an optical feature map related to target saliency extracted from the input image, such as brightness channel, specific color channel, edge intensity map, etc. Tanh is the hyperbolic tangent function. is the feature enhancement function; The formula for multi-scale output fusion is: ； in, is the gradient sensitivity coefficient, For multi-scale output fusion, is the feature map gradient.

8. A device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of any one of the deep learning-based underwater image target detection methods as claimed in claims 1 to 7.

9. A medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method for underwater image target detection based on deep learning as described in any one of claims 1 to 7 is implemented.

Images