Side scan sonar image denoising method based on improved ISTA-Net++ network

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a side-scan sonar image denoising method based on an improved ISTA‑Net++ network. The method comprises: (1) constructing a real sonar data set and a simulated sonar data set; (2) constructing a side-scan sonar image denoising network model by improving the DPMM module in the ISTA‑Net++ network; the improved DPMM module uses CoordConv convolution for feature extraction, and uses a residual attention module for feature processing, and an edge enhancement attention module is introduced into the residual attention module to enhance the edge of the target; (3) the network is trained using the Adam algorithm to obtain a trained side-scan sonar image denoising network model; (4) the real sonar data set is input into the trained side-scan sonar image denoising network model to obtain a clear side-scan sonar image after denoising. The method provided by the present invention reduces the loss of image quality and edge details while removing noise.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a side-scan sonar image denoising method based on an improved ISTA-Net++ network. Background Art

[0002] Acoustic signals are the only form of energy that can propagate over long distances in the ocean. Sonar technology is an important means of acquiring, utilizing and processing ocean information. The development of sonar technology is of great significance to the development of marine resources, marine environmental protection, and marine disaster warning. Among them, side-scan sonar has the characteristics of low cost, high resolution, and long detection distance. It is widely used in object detection in marine emergency investigations, marine ecology, marine archaeology, marine search and rescue, underwater structure identification, and damage detection.

[0003] However, affected by the imaging mechanism of Side Scan Sonar (SSS) and the complex underwater environment, the original SSS data is usually accompanied by strong noise, mainly in the form of speckle noise. In addition, the sound wave intensity is weakened during propagation due to the uneven distribution of seawater temperature and pressure, and the sound wave energy loss caused by the attenuation of the backscattered signal. The SSS system often performs time-varying gain (TVG) to compensate for the transmission loss of the target and restore the target signal. Since the TVG process is proportional to the propagation distance, the SSS image noise is non-uniform.

[0004] Most existing SSS image denoising methods rely on traditional filtering techniques or single deep learning models. These methods are usually unable to effectively process the uneven speckle noise in the image, resulting in blurred edge details and loss of structural information in the denoised image, which in turn affects the detection and recognition accuracy of underwater targets. In addition, most traditional methods lack the use of spatial position information and are unable to fully utilize the spatial structural characteristics of the image to enhance the denoising effect. Summary of the invention

[0005] In view of the above technical problems, the present invention provides a side-scan sonar image denoising method based on an improved ISTA-Net++ network, which can reduce the loss of image quality and edge details while removing noise.

[0006] The present invention is achieved through the following technical solutions:

[0007] The side scan sonar image denoising method based on the improved ISTA-Net++ network includes the following steps:

[0008] (1) Constructing real sonar datasets and simulated sonar datasets;

[0009] (2) Using the simulated sonar dataset as input, a side scan sonar image denoising network model (i.e., SSS image denoising algorithm, SIDA network) was constructed by improving the DPMM module (i.e., Dynamic Proximal Mapping Module) in the ISTA-Net++ network.

[0010] The improved DPMM module uses CoordConv convolution for feature extraction and residual attention module for feature processing, and an edge enhancement attention module is introduced into the residual attention module to enhance the target edge; wherein, the improved ISTA-Net++ is used for compressed sensing;

[0011] (3) Use the Adam algorithm to train the network and obtain the trained side-scan sonar image denoising network model;

[0012] (4) Inputting the real sonar data set into the trained side-scan sonar image denoising network model to obtain a clear side-scan sonar image after denoising.

[0013] Furthermore, in step (1), the method for constructing the simulated sonar data set is: converting the optical image into a grayscale image, and adding speckle noise with a standard deviation ranging from 18 to 45 to the grayscale image, to generate a noise image with non-uniform noise to simulate a real sonar image.

[0014] Furthermore, in step (2), the improved DPMM module includes: two residual attention modules , Feature extraction module , Image reconstruction module ; Feature extraction module , Image reconstruction module They are all composed of convolutional layers and ReLU;

[0015] The improved DPMM module is expressed as:

[0016] ;

[0017] in, represents the output of the kth stage of the improved DPMM module; R K Represents the input of the improved DPMM module, Indicates the condition information transmitted by the condition module CM to the improved DPMM module; is the function used to define the improved DPMM module in the kth stage; is an additional noise level map, the output from the conditioning module CM.

[0018] Furthermore, in step (2), CoordConv convolution is used for feature extraction, and CoordConv convolution concatenates the coordinate information with the input feature map.

[0019] Furthermore, in step (2), the improved DPMM module adopts two consecutive residual attention modules, each of which contains two dilated convolutional layers, two consecutive 3×3 convolutional layers and an edge-enhanced attention module.

[0020] Furthermore, in step (2), the edge-enhanced attention module first performs a 1×1 convolution on the input feature map to complete the data transformation from the input layer to the hidden layer of the IRNN (i.e., the Identity Matrix RNN); then, four IRNNs are used in four independent directions to obtain corresponding context features, and the results are fused into a feature map through a Concat operation.

[0021] Furthermore, in step (3), the network is trained using the Adam optimizer, given a training data set and the sampling matrix , and the sampling result is ,Will , , As input, to reduce the clean image And the denoised image The difference between , the loss function is calculated as:

[0022] ;

[0023] in, represents the total number of training sets, represents the number of sampling matrices, is the compressed sensing ratio; θ Represents the set of learnable parameters in the network.

[0024] Furthermore, in step (4), each original noisy image in the real sonar data set is directly input into the trained side-scan sonar image denoising network model, and the trained side-scan sonar image denoising network model is used to extract multi-level attention features to obtain a reconstructed side-scan sonar image.

[0025] Furthermore, the method further comprises step (5): using an image quality evaluation index to evaluate the side scan sonar image denoising network model.

[0026] Furthermore, the denoised images were evaluated using the full reference image evaluation indicators PSNR and SSIM and the no reference image evaluation indicators ENL and EPI.

[0027] Beneficial technical effects of the present invention:

[0028] The present invention improves the image denoising performance by improving the reconstruction part of the ISTA-Net++ algorithm. By using CoordConv to extract features in the image reconstruction network, the vertical coordinate channel is added to the pixel input channel to give it the property of non-uniform noise; then the multi-scale residual attention module is used to learn image features to improve the image reconstruction quality. The Adam optimizer is used to train the network, which effectively solves the problem of removing sonar image noise while better retaining image edge detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The improved DPMM module structure in the embodiment of the present invention;

[0030] Figure 2 This is a flow chart of feature extraction performed by a feature extraction module in an embodiment of the present invention;

[0031] Figure 3 This is a structural diagram of the residual attention module in an embodiment of the present invention;

[0032] Figure 4 Schematic diagram of image edge change in an embodiment of the present invention;

[0033] Figure 5 A schematic diagram showing that context information close to an image edge continues to propagate forward in an embodiment of the present invention;

[0034] Figure 6 This is a structural diagram of an edge-enhanced attention module in an embodiment of the present invention;

[0035] Figure 7 : is a performance comparison diagram of different compressed sensing sampling ratios in an embodiment of the present invention;

[0036] Figure 8 A visual comparison diagram of the method of the present invention and other denoising algorithms in simulating sonar image denoising;

[0037] Fig. 9 This is a visual comparison chart of the denoising of real sonar images using the method of the present invention and other denoising algorithms. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0040] The side scan sonar image denoising method based on the improved ISTA-Net++ network includes the following steps:

[0041] (1) Constructing real sonar datasets and simulated sonar datasets;

[0042] (2) Using the simulated sonar dataset as input, a side scan sonar image denoising network model (i.e., SSS image denoising algorithm, SIDA network) was constructed by improving the DPMM module (i.e., Dynamic Proximal Mapping Module) in the ISTA-Net++ network.

[0043] The improved DPMM module uses CoordConv convolution for feature extraction and residual attention module for feature processing, and an edge enhancement attention module is introduced into the residual attention module to enhance the target edge; wherein, the improved ISTA-Net++ is used for compressed sensing;

[0044] (3) Use the Adam algorithm to train the network and obtain the trained side-scan sonar image denoising network model;

[0045] (4) Inputting the real sonar data set into the trained side-scan sonar image denoising network model to obtain a clear side-scan sonar image after denoising.

[0046] In step (1) of the present invention, the method for constructing the simulated sonar data set is: converting the optical image into a grayscale image, and adding speckle noise with a standard deviation ranging from 18 to 45 to the grayscale image, to generate a noise image with non-uniform noise to simulate a real sonar image.

[0047] Specifically, the optical image is a clear optical picture in ImageNet.

[0048] In step (2) of the present invention, Figure 1 As shown, the improved DPMM module includes: two residual attention modules , Feature extraction module , Image reconstruction module ; Feature extraction module , Image reconstruction module They are all composed of convolutional layers and ReLU;

[0049] The improved DPMM module is expressed as:

[0050] ;

[0051] in, represents the output of the kth stage of the improved DPMM module; R K Represents the input of the improved DPMM module, Indicates the condition information transmitted by the condition module CM to the improved DPMM module; is the function used to define the improved DPMM module in the kth stage; is an additional noise level map, the output from the conditioning module CM.

[0052] In step (2) of the present invention, CoordConv convolution is used for feature extraction, and CoordConv convolution concatenates the coordinate information with the input feature map. This step can help the neural network better learn the spatial relationship in the input data. The whole process consists of two steps. The first step is to connect the input data with the additional channel containing the coordinate information to form combined data; the second step is to convolve the combined data through a convolution layer. Figure 2 As shown, the vertical coordinate channel is added to the pixel input channel, so that the extracted features also have non-uniform noise properties.

[0053] In step (2) of the present invention, the improved DPMM module uses two consecutive residual attention modules, each of which includes two dilated convolutional layers, two consecutive 3×3 convolutional layers, and an edge enhancement attention module. First, two dilated convolutional layers are applied to expand the receptive field of the input features to capture a wider range of contextual information; then the dilated convolutional features are connected to another convolutional layer to further process and extract richer feature representations; then two consecutive convolutional layers are used to deeply learn the abstract features of the image through residual connections, and the edge-enhanced attention mechanism is introduced to help retain the texture details of the reconstructed image and improve the computational efficiency and learning speed of the model. The structure of the residual attention module is shown in Figure 1. Figure 3 shown.

[0054] In the present invention, the edge-enhanced attention module uses IRNN to process two-dimensional sonar image data; four IRNNs are used to enhance single-point features, and the four IRNNs represent four basic directions: up, down, left, and right. Figure 4 As shown in , the context information in the horizontal direction is easier to identify the target than the context information in the vertical direction. Figure 5As shown, the context information close to the edge continues to propagate forward, which can enhance the features of the points inside the edge;

[0055] make is the position in the input feature map The value of the pixel at represents the corresponding output after IRNN propagation, and the update process can be expressed as:

[0056] ;

[0057] in, Represents the weight coefficient of the corresponding direction of IRNN, Represents the corresponding output after IRNN propagation; Indicates the output value of the previous position; is the position in the input feature map The weight coefficients of the IRNN are initialized to the identity matrix and updated through training.

[0058] In step (2), if Figure 6 As shown in the figure, the edge-enhanced attention module first performs a 1×1 convolution on the input feature map to complete the data transformation from the IRNN input layer to the hidden layer; then, four IRNNs are used in four independent directions to obtain the corresponding context features, and the results are fused into a feature map through a Concat operation.

[0059] In step (3) of the present invention, the network is trained using the Adam optimizer, given a training data set and the sampling matrix , and the sampling result is ,Will , , As input, to reduce the clean image And the denoised image The difference between , the loss function is calculated as:

[0060] ;

[0061] in, represents the total number of training sets, represents the number of sampling matrices, is the compressed sensing ratio; θ Represents the set of learnable parameters in the network.

[0062] In step (4) of the present invention, each original noisy image in the real sonar data set is directly input into the trained side-scan sonar image denoising network model, and the trained side-scan sonar image denoising network model is used to extract multi-level attention features to obtain a reconstructed side-scan sonar image.

[0063] In the present invention, the method further comprises step (5): using an image quality evaluation index to evaluate the side scan sonar image denoising network model.

[0064] In the present invention, the full reference image evaluation indexes PSNR and SSIM and the no-reference image evaluation indexes ENL and EPI are used to evaluate the denoised image.

[0065] Figure 7 The performance comparison of different compressed sensing sampling ratios is shown. The best performance is achieved when the sampling ratio is 50%, so the 50% sampling ratio is used in the remaining experiments for comparison with other models. Figure 8 A visual comparison diagram of the present invention and other denoising algorithms in simulating sonar image denoising; Fig. 9 This is a visual comparison chart of the denoising of real sonar images between the present invention and other denoising algorithms. Figure 8 and Fig. 9 The proposed image is denoised using the method provided by the present invention. It can be seen that compared with other traditional denoising algorithms (BM3D, K-SVD, Masked-Denoising, NAF-Net, NLM, ISTA, ISTA+, ISTA++), the method provided by the present invention effectively solves the problem of removing sonar image noise while better retaining image edge detail information.

[0066] The above embodiments are only for illustrating the inventive concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A side-scan sonar image denoising method based on an improved ISTA-Net++ network is characterized in that: The following steps are involved: (1) Construct real sonar datasets and simulated sonar datasets; (2) Taking the simulated sonar dataset as input, a side-scan sonar image denoising network model was constructed by improving the DPMM module in the ISTA-Net++ network; The improved DPMM module uses CoordConv convolution for feature extraction and a residual attention module for feature processing, and an edge enhancement attention module is introduced into the residual attention module to enhance the target edge; (3) Using the Adam algorithm to train the network, a trained side-scan sonar image denoising network model is obtained; (4) inputting the real sonar data set into the trained side-scan sonar image denoising network model to obtain a clear side-scan sonar image after denoising; In step (2), CoordConv convolution is used for feature extraction, and CoordConv convolution concatenates the coordinate information with the input feature map; the whole process consists of two steps. The first step is to connect the input data with the additional channel containing the coordinate information to form combined data; the second step is to convolve the combined data through a convolution layer; the vertical coordinate channel is added to the pixel input channel, so that the extracted features have non-uniform noise properties; In step (2), the improved DPMM module uses two consecutive residual attention modules, each of which contains two dilated convolutional layers, two consecutive 3×3 convolutional layers, and an edge-enhanced attention module; In step (2), the edge-enhanced attention module uses IRNN to process the two-dimensional sonar image data; four IRNNs are used to enhance single-point features, and the four IRNNs represent four basic directions: up, down, left, and right; Let h i,j is the value of the pixel at position (i, j) in the input feature map, f i,j represents the corresponding output after IRNN propagation, and the update process is expressed as: f i,j =max(α right f i,j-1 +h i,j ,0); Among them, α right Represents the weight coefficient of the corresponding direction of IRNN, f i,j represents the corresponding output after IRNN propagation; f i,j-1 Indicates the output value of the previous position; h i,j is the value of the pixel at position (i, j) in the input feature map; the weight coefficient of IRNN is initialized to the unit matrix and updated through training; The edge-enhanced attention module first performs a 1×1 convolution on the input feature map to complete the data transformation from the IRNN input layer to the hidden layer; then uses 4 IRNNs in four independent directions to obtain the corresponding context features, and fuses the results into a feature map through a Concat operation.

2. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 1, characterized in that: In step (1), the method for constructing the simulated sonar data set is: converting the optical image into a grayscale image, and adding speckle noise with a standard deviation ranging from 18 to 45 to the grayscale image to generate a noise image with non-uniform noise to simulate a real sonar image.

3. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 1, characterized in that: In step (2), the improved DPMM module includes: two residual attention modules Feature extraction module Image reconstruction module Feature extraction module Image reconstruction module They are all composed of convolutional layers and ReLU; The improved DPMM module is expressed as: in, represents the output of the kth stage of the improved DPMM module; R K represents the input of the improved DPMM module, σ K Represents the condition information transmitted from the condition module CM to the improved DPMM module; H K DPMM is the function used to define the improved DPMM module in the kth stage; is an additional noise level map, the output from the conditioning module CM.

4. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 1, characterized in that: In step (3), the network is trained using the Adam optimizer, given a training data set and the sampling matrix Get the sampled result Y i,t , Y i,t ,Φ t ,γ t As input, to reduce the clean image X i And the denoised image H(Y i,t ,Φ t ,γ t ), the loss function is calculated as: Among them, N D Represents the total number of training sets, N γ represents the number of sampling matrices, Υ is the compressed sensing ratio; θ represents the set of learnable parameters in the network.

5. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 4 is characterized in that: In step (4), each original noisy image in the real sonar data set is directly input into the trained side-scan sonar image denoising network model, and the trained side-scan sonar image denoising network model is used to extract multi-level attention features to obtain a reconstructed side-scan sonar image.

6. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 1, characterized in that: The method also includes step (5): using an image quality evaluation index to evaluate the side scan sonar image denoising network model.

7. The side scan sonar image denoising method based on the improved ISTA-Net++ network according to claim 6 is characterized in that: The denoised images are evaluated using the full reference image evaluation indicators PSNR and SSIM and the no reference image evaluation indicators ENL and EPI respectively.

Images