Sonar image target detection method based on deep learning, electronic equipment and storage medium

Abstract

The invention discloses a deep learning-based sonar image target detection method, electronic equipment and a storage medium, and belongs to the technical field of underwater target recognition. In order to improve the recognition accuracy of an underwater target sonar image, the method comprises the steps of collecting a side-scan sonar image; constructing a self-adaptive mixed median filtering method, and carrying out image denoising processing on the side-scan sonar image; performing image enhancement processing by adopting a Retinex method to obtain a side-scan sonar image after image enhancement processing; according to the method, improvement is carried out based on a YOLOv5 model, a CBAM attention mechanism module is added on the basis of the YOLOv5 model, a bidirectional feature pyramid network is used to replace an FPN + PAN structure, and a deep learning-based sonar image target detection model is obtained; and inputting the side-scan sonar image after image enhancement processing into a deep learning-based sonar image target detection model to carry out deep learning-based sonar image target detection. According to the invention, the recognition accuracy of the underwater target sonar image is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target recognition, and specifically relates to a sonar image target detection method based on deep learning, an electronic device and a storage medium. Background Art

[0002] The ocean is an area that has not yet been developed on a large scale, especially the complex environment of the seabed and the rich and diverse natural resources in the deep sea. These factors have prompted countries around the world to invest a lot of manpower, material resources and financial resources in ocean exploration, especially deep-sea exploration and deep-sea operations research.

[0003] Underwater target detection primarily relies on underwater target images. Depending on the method used to acquire these images, these methods can be categorized as optical or acoustic. Optical images offer high resolution, realistic target depictions, and more detailed features. Researchers have been conducting underwater target detection based on optical images since the 1990s. However, due to the significant energy attenuation of light propagating underwater, long-distance underwater target detection presents challenges.

[0004] Acoustic imaging primarily utilizes sonar to image underwater targets. Sonar offers advantages such as long transmission range, wide imaging field of view, and accurate target positioning. Therefore, acoustic imaging is often the primary method of choice for underwater target detection. The underwater environment is complex and dynamic, and the quality of available underwater data is poor. In shallow waters or near coastal environments, received signals often contain significant amounts of ambient noise, reverberation noise, and self-noise. Consequently, acoustic detection and recognition of underwater targets still present numerous challenges. Therefore, it is necessary to design a sonar image-based approach that applies deep learning algorithms to underwater target detection tasks to address these technical challenges. Summary of the Invention

[0005] The problem to be solved by the present invention is to improve the recognition accuracy of underwater target sonar images, and propose a sonar image target detection method, electronic equipment and storage medium based on deep learning.

[0006] To achieve the above object, the present invention is implemented through the following technical solutions:

[0007] A sonar image target detection method based on deep learning includes the following steps:

[0008] S1. Collect side-scan sonar images;

[0009] S2. Constructing an adaptive hybrid median filtering method to perform image denoising on the side-scan sonar image collected in step S1;

[0010] S3. Perform image enhancement processing on the denoised side-scan sonar image obtained in step S2 using the Retinex method to obtain an enhanced side-scan sonar image;

[0011] S4. Improve the YOLOv5 model by adding the CBAM attention mechanism module and replacing the FPN+PAN structure with a bidirectional feature pyramid network to obtain a deep learning-based sonar image target detection model.

[0012] S5. Input the side-scan sonar image after image enhancement processing obtained in step S3 into the deep learning-based sonar image target detection model obtained in step S4 to perform deep learning-based sonar image target detection.

[0013] Furthermore, the adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on the local gradient modulus to achieve dynamic filtering of noise-sensitive areas; the calculation formula of the local gradient modulus is:

[0014]

[0015] Among them, M(x,y) is the local gradient modulus, G x is the improved Sobel horizontal gradient operator, G y is the vertical gradient operator, α is the horizontal adjustment coefficient, β is the vertical adjustment coefficient, and γ is the cross-term compensation factor;

[0016] The calculation formula of the adaptive weight function is:

[0017]

[0018] Among them, w(x,y) is the adaptive weight function, σ is the adjustment factor, tanh is the hyperbolic tangent function, and k is the curvature parameter;

[0019] The expression for calculating the improved median is:

[0020]

[0021] in, sort is to sort the pixel values in the filter window in ascending order and then take the median value, w i is the adaptive weight, p i is the pixel value preprocessed by bilateral filtering, Δp is the domain feature difference, τ is the nonlinear attenuation coefficient, i is any one of n, and n is the total summation number.

[0022] Furthermore, the Retinex method in step S3 is a single-scale Retinex method.

[0023] Furthermore, step S4 adds a CBAM attention mechanism module based on the YOLOv5 model, and uses global average pooling and standard deviation pooling operations to calculate the attention maps of the spatial feature map and the channel feature map respectively. Then, the spatial attention score and the channel attention score are obtained through the sigmoid function. Finally, the obtained spatial attention score and channel attention score are multiplied with the original feature map to realize the weighting of the feature map.

[0024] Furthermore, the formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is:

[0025]

[0026] Among them, w1, w2 are the first normalized fusion weight and the second normalized fusion weight respectively, and u is the upsampling operation. is the downsampling operation, is the output of the previous layer of layer l, is the next layer output of layer l, is the input feature map of the lth layer;

[0027] The formula of the feature enhancement function is:

[0028]

[0029] Among them, E s is the sonar echo intensity characteristic diagram, tanh is the hyperbolic tangent function, F enhanced is the feature enhancement function;

[0030] The formula for multi-scale output fusion is:

[0031]

[0032] Among them, σ is the gradient sensitivity coefficient, Output is the multi-scale output fusion, is the feature map gradient of layer l.

[0033] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a sonar image target detection method based on deep learning when executing the computer program.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the sonar image target detection method based on deep learning.

[0035] Beneficial effects of the present invention:

[0036] The deep learning-based sonar image target detection method described in the present invention uses a new median filtering algorithm for denoising. The peak signal-to-noise ratio, mean square error, and structural similarity are all superior compared to other filtering methods.

[0037] The deep learning-based sonar image target detection method described in this paper utilizes the Retinex algorithm to mitigate or eliminate image quality issues caused by uneven lighting, improving overall image visibility. It also highlights details in the image, making both dark and bright areas more clearly visible.

[0038] The deep learning-based sonar image target detection method described in this paper improves the BiFPN architecture. Residual connections are added to the model to enhance feature expression capabilities; single-input edge nodes are removed, significantly reducing the number of model parameters and accelerating model inference speed; and weights are added to each scale feature during the fusion process, and the contribution of each scale is adjusted to achieve weighted fusion of different scales.

[0039] The deep learning-based sonar image target detection method described in this paper incorporates a CBAM attention mechanism module. This module employs two levels of attention: channel attention and spatial attention, capturing the interdependencies between channels and the strong correlations in spatial dimensions, respectively. Consequently, the CBAM attention mechanism enables the network to focus on important channels and spatial locations for feature expression, effectively improving feature quality and expressiveness. Dynamic sparse gating is introduced on top of CBAM to reduce computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of a sonar image target detection method based on deep learning described in the present invention. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present invention and are not intended to limit the present invention. That is, the specific embodiments described herein are only some embodiments of the present invention, not all embodiments. Generally, the components of the specific embodiments of the present invention described and illustrated in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0042] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected specific embodiments of the present invention. All other specific embodiments obtained by those skilled in the art based on the specific embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] In order to further understand the content, features and effects of the present invention, the following specific embodiments are given as examples, and the attached Figure 1 The detailed instructions are as follows:

[0044] Example 1:

[0045] A sonar image target detection method based on deep learning includes the following steps:

[0046] S1. Collect side-scan sonar images;

[0047] S2. Constructing an adaptive hybrid median filtering method to perform image denoising on the side-scan sonar image collected in step S1;

[0048] Furthermore, the adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on the local gradient modulus to achieve dynamic filtering of noise-sensitive areas; the calculation formula of the local gradient modulus is:

[0049]

[0050] Among them, M(x,y) is the local gradient modulus, G x is the improved Sobel horizontal gradient operator, G y is the vertical gradient operator, α is the horizontal adjustment coefficient, β is the vertical adjustment coefficient, and γ is the cross-term compensation factor;

[0051] The calculation formula of the adaptive weight function is:

[0052]

[0053] Among them, w(x,y) is the adaptive weight function, σ is the adjustment factor, tanh is the hyperbolic tangent function, and k is the curvature parameter;

[0054] The expression for calculating the improved median is:

[0055]

[0056] in, sort is to sort the pixel values in the filter window in ascending order and then take the median value, w i is the adaptive weight, p iis the pixel value preprocessed by bilateral filtering, Δp is the domain feature difference, τ is the nonlinear attenuation coefficient, i is any one of n, and n is the total summation number.

[0057] Furthermore, the median value of the pixel value within the filter window is combined with the left and right adjacent pixel values to form a combination and normalized. The normalized results are used as weights for the three pixel values, multiplied and summed, and the final sum is used to replace the pixel value at the center of the current filter window. The process is completed; the edge-preserving denoising effect is achieved through weighted median sorting. A cross-term compensation factor γ is introduced into the gradient modulus calculation to enhance the oblique edge response characteristics. The weight function uses a hyperbolic tangent nonlinear transformation, and domain adaptive adjustment is achieved through the curvature parameter k. The median calculation integrates weighted sorting and nonlinear attenuation terms to optimize the denoising effect in texture areas while preserving edges.

[0058] S3. Perform image enhancement processing on the denoised side-scan sonar image obtained in step S2 using the Retinex method to obtain an enhanced side-scan sonar image;

[0059] Furthermore, the Retinex method in step S3 is a single-scale Retinex method.

[0060] Furthermore, image enhancement, when calculating the grayscale value of a target pixel in an image, is obtained by weighting the pixel values in the area centered on the target point, and the weight ratio is determined by the surround function;

[0061] The steps of the single-scale Retinex method are:

[0062] Step 1. Convert the data type of the denoised side-scan sonar image S(x, y) obtained in step S2 from integer to double.

[0063] Step 2. Determine the smoothness c of the scale parameter Gaussian filter and the weight coefficient λ. The recommended range of c is 15 to 200, and λ is set to 1.

[0064] Step 3. Calculate the value of r(x, y);

[0065] r(x,y)=log(S(x,y)+ε)-log(F c (x,y)*S(x,y)+ε)

[0066] Among them, F c (x, y) is a Gaussian filter with a kernel size determined by c, used to estimate the illumination component; ε is the minimum value, set to 10 -6 , to avoid taking the logarithm of zero;

[0067] Step 4. Transform r(x, y) from the logarithmic domain to the real domain R(x, y), the expression is:

[0068] R(x,y)=e r(x,y)

[0069] Step 5. Perform linear correction on R(x, y). The image enhancement result is obtained after correction. The linear correction formula is as follows:

[0070]

[0071] S4. Improve the YOLOv5 model by adding the CBAM attention mechanism module and replacing the FPN+PAN structure with a bidirectional feature pyramid network to obtain a deep learning-based sonar image target detection model.

[0072] Furthermore, Python programming language and NumPy library are used to implement time series modeling; further,

[0073] Furthermore, step S4 adds a CBAM attention mechanism module based on the YOLOv5 model, and uses global average pooling and standard deviation pooling operations to calculate the attention maps of the spatial feature map and the channel feature map respectively. Then, the spatial attention score and the channel attention score are obtained by the sigmoid function. Finally, the obtained spatial attention score and channel attention score are multiplied with the original feature map to achieve feature map weighting;

[0074] Furthermore, dynamic sparse gating is introduced on the basis of CBAM to reduce the computational complexity. The formula of dynamic sparse attention mechanism is as follows:

[0075] The formula for channel attention enhancement is:

[0076] M c (F)=σ(MLP([AvgPool(F);StdPool(F)]))

[0077]

[0078] Among them, H is the feature map height (Height), W is the feature map width (Width), F is the input feature map (Featuremap), F ij is the eigenvalue of the feature map at the spatial position (i, j), μ F is the global average value of the feature map, MLP is a multi-layer perceptron (Multi-Layer Perceptron), σ is a Sigmoid activation function; feature motion information is captured by adding standard deviation pooling.

[0079] The formula for spatial attention improvement is:

[0080] M s (F)=σ(f 7×7 ([F max ; F avg ]))☉G(F)

[0081] G(F) = sigmoid(θ·Conv(F))

[0082] Among them, F is the input feature map, the dimension is C×H×W (number of channels×height×width), F max is the maximum pooling result of the feature map in the spatial dimension (H×W), dimension C×1×1, F avg is the average pooling result of the feature map in the spatial dimension, with a dimension of C×1×1, [;] is the splicing operation along the channel dimension (if the original number of channels is C, the number after splicing is 2C); f 7×7 is a convolutional layer containing a 7×7 convolution kernel, used to fuse spatial information; σ is a Sigmoid activation function that maps weights to the interval [0,1]. Conv(F) is the convolution operation on the feature map F; θ is a learnable sparsification gating parameter used to adjust the sparsity of weights at each spatial position; ⊙ is the element-wise multiplication (Hadamard product);

[0083] Furthermore, the formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is:

[0084]

[0085] Among them, w1 and w2 are the first normalized fusion weight and the second normalized fusion weight respectively. is the upsampling operation, is the downsampling operation, is the output of the previous layer of layer l, is the next layer output of layer l, is the input feature map of the lth layer;

[0086] The formula of the feature enhancement function is:

[0087]

[0088] Among them, E s is the sonar echo intensity characteristic diagram, tanh is the hyperbolic tangent function, F enhanced is the feature enhancement function;

[0089] The formula for multi-scale output fusion is:

[0090]

[0091] Among them, σ is the gradient sensitivity coefficient, Output is the multi-scale output fusion, is the feature map gradient of layer l.

[0092] Furthermore, the deep learning-based sonar image target detection model can more accurately identify sonar image features, identify different types of sonar images through features, and provide more accurate image information for sonar equipment.

[0093] S5. Input the side-scan sonar image after image enhancement processing obtained in step S3 into the deep learning-based sonar image target detection model obtained in step S4 to perform deep learning-based sonar image target detection.

[0094] Example 2:

[0095] This embodiment differs from the first embodiment in that the median filtering algorithm calculation process of this embodiment is as follows: first, the sonar image is grayscaled, and the pixel values of the three channels R, G, and B are multiplied with different weights and summed to obtain the grayscale value. The calculation formula is:

[0096] f(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,,y

[0097] Where (x, y) represents the coordinates of the current pixel point, f(x, y) represents the pixel value after grayscale processing, R(x, y) represents the red component value, G(x, y) represents the green component value, and B(x, y) represents the blue component value.

[0098] Place the current 3×3 filter window in the upper left corner of the original image; determine the pixel values in the current filter window in order from left to right and from top to bottom. The working principle of median filtering is to sort all pixels according to the size of pixel values in the local pixel composition area of a given image, and then take the pixel in the middle position after sorting as the new pixel value in the area.

[0099] Example 3:

[0100] The difference between this embodiment and embodiment 1 is that the calculation formula of the attention mechanism of this embodiment is as follows:

[0101]

[0102] Among them, Query is the query vector, Source is the input sequence, including L x Elements; L x is the length of the input sequence (i.e. ||Source||, indicating the dimension or number of elements of the sequence); Key iis the key vector of the i-th input element; Value i is the value vector of the i-th input element, Similarity(Query,Key i ) is the similarity between the query vector and the key vector;

[0103] First, the correlation between the query and different keys is calculated, that is, the weight coefficients of different values are calculated; then the output of the previous stage is normalized to map the range of values between 0 and 1; finally, the values are weighted and summed according to the weight coefficients to obtain the final attention value.

[0104] Example 4:

[0105] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the sonar image target detection method based on deep learning described in Example 1 are implemented.

[0106] The computer device of the present invention may include a processor and memory, such as a single-chip microcomputer including a central processing unit. Furthermore, the processor is configured to execute a computer program stored in the memory to implement the steps of the aforementioned deep learning-based sonar image target detection method.

[0107] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0108] The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); and the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0109] Example 5:

[0110] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for sonar image target detection based on deep learning described in Example 1 is implemented.

[0111] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned sonar image target detection method based on deep learning can be implemented.

[0112] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0113] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0114] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and components may be substituted with equivalents without departing from the scope of the present application. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of these combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions within the scope of the claims.

Claims

1. A sonar image target detection method based on deep learning, characterized in that: The steps include: S1. Collect side-scan sonar images; S2. Constructing an adaptive hybrid median filtering method to perform image denoising on the side-scan sonar image collected in step S1; S3. Perform image enhancement processing on the denoised side-scan sonar image obtained in step S2 using the Retinex method to obtain an enhanced side-scan sonar image; S4. Improve the YOLOv5 model by adding the CBAM attention mechanism module and replacing the FPN+PAN structure with a bidirectional feature pyramid network to obtain a deep learning-based sonar image target detection model. S5. Input the side-scan sonar image after image enhancement processing obtained in step S3 into the deep learning-based sonar image target detection model obtained in step S4 to perform deep learning-based sonar image target detection.

2. The sonar image target detection method based on deep learning according to claim 1, characterized in that: The adaptive hybrid median filtering method in step S2 is an adaptive weight adjustment mechanism based on the local gradient modulus to achieve dynamic filtering in noise-sensitive areas. The calculation formula of the local gradient modulus is: Among them, M(x,y) is the local gradient modulus, G x is the improved Sobel horizontal gradient operator, G y is the vertical gradient operator, α is the horizontal adjustment coefficient, β is the vertical adjustment coefficient, and γ is the cross-term compensation factor; The calculation formula of the adaptive weight function is: Among them, w(x,y) is the adaptive weight function, σ is the adjustment factor, tanh is the hyperbolic tangent function, and k is the curvature parameter; The expression for calculating the improved median is: in, sort is to sort the pixel values in the filter window in ascending order and then take the median value, w i is the adaptive weight, p i is the pixel value preprocessed by bilateral filtering, Δp is the domain feature difference, τ is the nonlinear attenuation coefficient, i is any one of n, and n is the total summation number.

3. The sonar image target detection method based on deep learning according to claim 2, characterized in that: The Retinex method in step S3 is a single-scale Retinex method.

4. The sonar image target detection method based on deep learning according to claim 3, characterized in that: Step S4 adds the CBAM attention mechanism module based on the YOLOv5 model, and uses global average pooling and standard deviation pooling operations to calculate the attention maps of the spatial feature map and the channel feature map respectively. Then, the spatial attention score and the channel attention score are obtained through the sigmoid function. Finally, the obtained spatial attention score and channel attention score are multiplied with the original feature map to realize the weighting of the feature map.

5. The sonar image target detection method based on deep learning according to claim 4, characterized in that: The formula for bidirectional feature fusion in the bidirectional feature pyramid network in step S4 is: Among them, w1 and w2 are the first normalized fusion weight and the second normalized fusion weight respectively. is the upsampling operation, is the downsampling operation, is the output of the previous layer of layer l, is the next layer output of layer l, P l out is the input feature map of the lth layer; The formula of the feature enhancement function is: F enhanced =P l out ⊙tanh(log(1+E s )) Among them, E s is the sonar echo intensity characteristic diagram, tanh is the hyperbolic tangent function, F enhanced is the feature enhancement function; The formula for multi-scale output fusion is: Among them, σ is the gradient sensitivity coefficient, Output is the multi-scale output fusion, is the feature map gradient of layer l.

6. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the sonar image target detection method based on deep learning according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for sonar image target detection based on deep learning according to any one of claims 1 to 5 is implemented.

Images