Sonar image super-resolution method and system based on diffusion generative model
By adopting a sonar image super-resolution method based on a diffusion generation model, the artifact problem caused by noise and texture coupling is solved, and high-fidelity, globally consistent sonar image super-resolution reconstruction is achieved, which is suitable for underwater edge devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF SOFTWARE - CHINESE ACAD OF SCI
- Filing Date
- 2026-03-31
- Publication Date
- 2026-07-14
Smart Images
Figure CN121961848B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of underwater image processing technology, specifically relating to a sonar image super-resolution method and system based on a diffusion generation model. Background Technology
[0002] For underwater exploration missions, the quality of side-scan sonar images is often constrained by various degradation factors in the complex underwater acoustic channels. These factors collectively lead to images exhibiting low resolution, significant speckle noise, and blurred edge details. Traditional image super-resolution methods, such as interpolation algorithms and convolutional neural network-based methods, can improve image resolution to some extent, but they struggle to effectively model and reverse the complex physical degradation processes that sonar images undergo in real underwater environments. This results in distorted details and unnatural textures in reconstructed images, limiting their discrimination performance when dealing with complex seabed topography and small targets. Therefore, in sonar image super-resolution reconstruction, constructing a data model that can accurately fit the physical mechanisms of acoustic imaging and generate a data model that conforms to the distribution of real high-resolution images is crucial for improving the fidelity of reconstructed images, enhancing the practicality of algorithms, and promoting the intelligent development of underwater exploration.
[0003] Super-resolution sonar data has significant application value in the fields of marine exploration and underwater operations, including but not limited to: providing high-quality training samples for image restoration tasks such as sonar image denoising, deblurring, and contrast enhancement; supporting the construction of data augmentation and benchmark evaluation systems for advanced vision tasks such as underwater target recognition, seabed topography classification, and shipwreck detection; and achieving effective data expansion and model generalization capability improvement in practical application scenarios such as limited samples and cross-device migration.
[0004] In existing side-scan sonar image processing technologies, the main methods for sonar image super-resolution include traditional interpolation methods, methods based on convolutional neural networks (CNNs), and methods based on generative adversarial networks (GANs).
[0005] Traditional interpolation is a simple and direct image super-resolution method that increases image resolution by inserting new pixels between the original pixels. Common interpolation algorithms include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. These methods are easy to implement and have low computational complexity. However, traditional interpolation methods (such as bilinear and bicubic interpolation) rely solely on simple weighted estimation of missing values for local pixels, without considering the acoustic characteristics of side-scan sonar images (texture directionality, multiplicative noise distribution). This results in blurred edges and loss of key details (such as small target outlines and seabed textures) in the super-resolution (SR) results, while noise is amplified simultaneously. This makes it unsuitable for the image clarity requirements of subsequent underwater target detection and recognition, and difficult to adapt to practical scenarios such as marine engineering and underwater search and rescue.
[0006] In recent years, with the development of deep learning technology, super-resolution methods based on convolutional neural networks (CNNs) have gradually become a research hotspot. CNNs can effectively improve image resolution by learning low-level and high-level features of images. For example, models such as SRCNN, VDSR, and ESPCN extract image features and reconstruct high-resolution images step by step by stacking multiple convolutional layers. However, CNN-like methods (such as VDSR and RCAN) have limited receptive fields, making it difficult to capture global texture associations in side-scan sonar images (such as the continuity of large-scale seabed topography), resulting in poor global consistency of SR results; they rely on pixel-level loss (L1 / L2) optimization, which easily generates "over-smoothed" images, losing key details and textures of sonar images; they are not robust enough to non-uniform noise, and are prone to mislearning noise as valid texture during training, producing artifacts; and they have weak generalization ability in small sample scenarios, making it difficult to adapt to different underwater acoustic environments.
[0007] Generative Adversarial Networks (GANs), as powerful generative models, have been widely used in the field of image super-resolution. GANs, by introducing a discriminator network and a generator network for adversarial training, can generate more realistic and higher-quality images. For example, models such as SRGAN and ESRGAN, through the design of specific generator and discriminator structures, effectively improve image resolution and detail recovery capabilities. However, adversarial training of GAN-like methods (such as ESRGAN and MHGAN) is prone to instability in small sample scenarios with side-scan sonar, generating "pseudo-details" with no real acoustic significance; the discriminator is sensitive to noise, easily forcing the generator to over-suppress noise and lose effective details; the model computational complexity is high, making it difficult to deploy in real-time on underwater edge devices (such as underwater robots), and thus failing to meet the requirements of real-time super-resolution.
[0008] In summary, existing methods struggle to handle artifacts caused by noise and texture coupling, as well as the large number of parameters in super-resolution models, slow reconstruction speed, and difficulty in adapting to underwater edge devices. Summary of the Invention
[0009] This invention provides a sonar image super-resolution method and system based on a diffusion generation model, which can realize the construction of training data for sonar acoustic degradation adaptation, the balance between detail restoration and global smoothness, the effective separation of noise and texture, and the improvement of super-resolution reconstruction efficiency, thereby obtaining high-fidelity, high signal-to-noise ratio, globally consistent, and real-time deployment-adapted high-resolution sonar images.
[0010] To achieve the above objectives, the technical solution of the present invention includes the following:
[0011] A sonar image super-resolution method based on a diffusion generation model, the method comprising:
[0012] The initial diffusion state is generated based on the input image and Gaussian noise that follows a standard normal distribution.
[0013] Input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state;
[0014] Based on the current diffusion state, the total noise is separated into structural noise and random noise, and then fused to obtain optimized noise;
[0015] Based on the optimized noise and the current diffusion state, calculate the preliminary estimated image corresponding to the current iteration time step;
[0016] Generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, generate the diffusion state for the next iteration time step;
[0017] Based on the diffusion state of the next iteration time step, the process of inputting the current diffusion state and the current iteration time step into the diffusion generation model is repeated until the inverse diffusion iteration process ends, resulting in a super-resolution image.
[0018] Furthermore, the process of generating the input image includes:
[0019] Acquire raw sonar images;
[0020] The original sonar image was denoised and its details were restored to obtain the input image.
[0021] Furthermore, based on a structured and random noise separator, the total noise is separated into structured noise and random noise by combining the current diffusion state; wherein, the structured and random noise separator includes an input layer, a multi-scale feature extraction layer, a cross-scale attention fusion layer, a dual-branch refined modeling layer, a feature reconstruction layer, and an output optimization layer;
[0022] The input layer is used to fuse the features of the current diffusion state and the total noise to obtain the first fused feature;
[0023] The multi-scale feature extraction layer is used to extract noise features of the first fused feature at different scales;
[0024] The cross-scale attention fusion layer is used to weightedly fuse noise features at different scales through an attention mechanism to obtain a second fused feature;
[0025] The dual-branch refined modeling layer is used to focus on the spatial correlation and sparsity of the second fusion feature, respectively, to obtain preliminary structural noise features and preliminary random noise features;
[0026] The feature reconstruction layer is used to upsample and fuse the preliminary structural noise features and preliminary random noise features respectively to obtain structural noise and random noise.
[0027] The output optimization layer is used to ensure that the structural noise and random noise of the output conform to the corresponding specific distribution constraints, including: Gamma distribution constraints or standard normal distribution.
[0028] Furthermore, the process of training the diffusion generation model and the structure-random noise separator includes:
[0029] Construct image pairs consisting of low-resolution images and high-resolution ground truth images;
[0030] A random iteration time step is sampled, and according to a preset noise schedule, a set noise is injected into the high-resolution ground truth image in proportion to generate a noisy image.
[0031] Using a noisy image, an iterative time step, and a low-resolution image as input to the diffusion generation model, and combining the structure and random noise separator, the total noise, structural noise, random noise, optimization noise, and current diffusion state of the noisy image are obtained.
[0032] The basic diffusion loss is obtained based on the mean square error between the set noise and the optimized noise;
[0033] By applying total variational regularization constraints to the structural noise, structural noise constraints are obtained.
[0034] Random noise constraints are obtained by imposing sparsity constraints on random noise;
[0035] Deep features of the preliminary estimated image and the high-resolution ground truth image corresponding to the current diffusion state are extracted by a pre-trained VGG19 network. The distance of the deep features in the feature space is calculated to obtain the perceptual consistency loss.
[0036] The total training loss is obtained based on the basic diffusion loss, structural noise constraint, random noise constraint, and perceptual consistency loss.
[0037] Backpropagation is performed based on the total training loss to update the parameters of the diffusion generation model and the structure and random noise separator.
[0038] Furthermore, the process of generating the low-resolution image includes:
[0039] Preprocessing is performed on high-resolution ground truth images, including: cropping, removing pure black pixels, image grayscale normalization, and tensor format conversion;
[0040] Based on the set acoustic propagation attenuation parameters, the preprocessed image is attenuated;
[0041] Based on the underwater suspended particles and turbulent scattering contained in the high-resolution ground truth image, a Gaussian blur kernel is constructed, and the attenuated image is convolved based on the Gaussian blur kernel to obtain the blurred image.
[0042] A Speckle noise map is generated based on the Gamma distribution, and the Speckle noise map is multiplied pixel by pixel with the blurred image to generate an image containing granular inherent noise.
[0043] Downsampling of images containing granular inherent noise;
[0044] The grayscale values of the downsampled image are converted to a grayscale image with a set number of bits to obtain a low-resolution image.
[0045] Furthermore, based on the optimized noise and the current diffusion state, the preliminary estimated image corresponding to the current iteration time step is calculated, including:
[0046] Generate a scaling factor based on the decay coefficient of the current iteration time step;
[0047] Calculate the weighted sum of the optimization noise and the current diffusion state based on the decay coefficient of the current iteration time step;
[0048] Based on the scaling factor and the weighted sum, a preliminary estimated image corresponding to the current reverse diffusion step is obtained.
[0049] Furthermore, based on the preliminary image estimation, noise optimization, and sampling noise, the diffusion state for the next iteration time step is generated, including:
[0050] Calculate the sampling standard deviation of the current reverse diffusion time step based on the decay coefficient of the current iteration time step and the decay coefficient of the previous reverse diffusion time step;
[0051] The first weight is obtained based on the decay coefficient of the preceding reverse diffusion time step;
[0052] The second weight is obtained based on the decay coefficient of the preceding reverse diffusion time step and the sampling standard deviation;
[0053] The third weight is obtained based on the sampling standard deviation;
[0054] The diffusion state of the preliminary estimated image, optimization noise, and sampling noise are calculated by weighting the first weight, the second weight, and the third weight, to obtain the diffusion state of the next iteration time step.
[0055] A sonar image super-resolution system based on a diffusion generation model, the system comprising:
[0056] An initial expansion state construction module is used to generate an initial diffusion state based on the input image and Gaussian noise that follows a standard normal distribution;
[0057] The noise prediction module is used to input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state;
[0058] The noise optimization module is used to combine the current diffusion state, separate the total noise into structural noise and random noise, and then fuse them to obtain optimized noise;
[0059] The preliminary estimated image generation module is used to calculate the preliminary estimated image corresponding to the current iteration time step based on the optimized noise and the current diffusion state;
[0060] The diffusion state update module is used to generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, to generate the diffusion state for the next iteration time step;
[0061] The super-resolution image output module is used to re-execute the diffusion generation model by inputting the current diffusion state and the current iteration time step based on the diffusion state of the next iteration time step, until the inverse diffusion iteration process ends, and obtain the super-resolution image.
[0062] A computer device, the computer device comprising: a processor and a memory storing computer program instructions; wherein the processor, when executing the computer program instructions, implements the sonar image super-resolution method based on the diffusion generation model described above.
[0063] A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the sonar image super-resolution method based on the diffusion generation model described above.
[0064] Compared with the prior art, the present invention has at least the following beneficial effects.
[0065] (1) This invention proposes a data preprocessing and attention guidance mechanism for acoustic degradation adaptation, which effectively solves the problems of image edge blurring and noise amplification caused by traditional methods. Specifically, in response to the problem that traditional interpolation and general deep learning methods cannot adapt to the unique acoustic degradation law of side-scan sonar, this invention constructs training data pairs that conform to the physical mechanism by simulating the energy attenuation and scattering process in underwater sound wave propagation. At the same time, an attention module is embedded in the network backbone, enabling the model to focus on key target structures and edges during reconstruction, thereby significantly improving the clarity and fidelity of the super-resolution results and avoiding the loss of details.
[0066] (2) This invention proposes a multi-loss weighted optimization texture fidelity mechanism, which effectively solves the acoustic texture distortion problem caused by excessive smoothing in CNN methods. Specifically, in response to the shortcomings of existing CNN-based methods that rely on pixel-level loss, resulting in "over-smoothing" of reconstructed images and loss of key textures, this invention innovatively adopts a multi-objective weighted optimization strategy that combines perceptual loss and noise prediction loss. This mechanism not only pursues pixel accuracy but also emphasizes the accuracy of matching deep semantic features and noise modeling, thereby ensuring the natural coherence of the global texture and high visual perceptual quality of the image while restoring high-resolution details.
[0067] (3) This invention proposes a mechanism for separating and specifically suppressing structural noise and random noise, effectively achieving decoupling and balancing speckle noise removal with the preservation of realistic texture details. Specifically, considering the spatial correlation of speckle noise in sonar images, this invention innovatively decomposes the predicted noise into two independent components, structural noise and random noise, in the core denoising step of the diffusion model. By modeling and differentiating the two branches separately, structural artifacts such as speckle can be accurately suppressed during the inverse diffusion process, while preserving or enhancing the realistic texture related to the target. This mechanism fundamentally avoids the excessive smoothing of details caused by the "one-size-fits-all" approach of traditional super-resolution methods, significantly improving the target recognition and practicality of the reconstructed image. Attached Figure Description
[0068] Figure 1 Flowchart of a sonar image super-resolution method based on a diffusion generation model.
[0069] Figure 2 Architecture diagram of the UNet backbone network.
[0070] Figure 3 Flowchart of data preprocessing for acoustic degradation adaptation of sonar images.
[0071] Figure 4 Block diagram of a sonar image super-resolution system based on a diffusion generation model.
[0072] Figure 5 A block diagram of computer equipment. Detailed Implementation
[0073] The present invention will be further described below with reference to possible accompanying drawings and specific embodiments, but this does not constitute any limitation on the present invention.
[0074] The sonar image super-resolution method based on the diffusion generation model of the present invention, such as Figure 1 As shown, the process includes steps S1 to S6.
[0075] Step S1: Generate the initial diffusion state based on the input image and Gaussian noise that follows a standard normal distribution.
[0076] This embodiment first generates a standard normal distribution based on Python's numpy.random.normal function. Gaussian noise The Gaussian noise Size and input image Consistent (256×256 pixels), used to simulate random perturbations during the initial noise addition process of the diffusion model.
[0077] Next, the total number of diffusion steps in the diffusion model was set to T=1000, and the attenuation coefficient corresponding to the total number of diffusion steps was obtained. ( For a predefined attenuation coefficient sequence The 1000th element (with a value of 0.001) is used to construct the initial diffusion state through "input image weighting + Gaussian noise weighting". The calculation formula is: ,in The image is a diffusion image in a chaotic state, with a size of 256×256 pixels.
[0078] Finally, the present invention can be based on this initial diffusion state. Initiate the reverse diffusion iteration loop (steps S2 to S5 below). In one embodiment, the present invention sets the reverse diffusion sampling step number N=50, and initiates a reverse diffusion iteration loop decreasing from N=50 to n=1 to gradually diffuse the initial chaotic state. Optimized to a clear super-resolution image. In each iteration, the current diffusion time step t and the time step s corresponding to the previous reverse diffusion step are calculated based on the current iteration number n and the total number of diffusion steps T. The calculation formulas are as follows: For example, when n=50, , .
[0079] In a preferred embodiment, the input image is obtained by denoising and detail restoration of the original sonar image. Specifically, this embodiment uses a low-quality original sonar image... Input CNN pre-trained restorer Through the multi-layer convolution and residual connections of CNN, Perform initial noise reduction and detail restoration to output an intermediate image with preliminary improved image quality. The calculation formula is: .in The resolution is the restored 256×256 pixels.
[0080] Step S2: Input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state.
[0081] The diffusion state of the current iteration step With time step t, the input diffusion generation model By extracting features from the UNet backbone network of the diffusion generation model and the sonar texture attention module, the total noise contained in the image under the current diffusion state is predicted. , Size and Consistent (256×256 pixels).
[0082] Figure 2 This is a diagram of the UNet backbone network architecture. The data processing process of this UNet backbone network is described below.
[0083] 1) Input preparation:
[0084] Input 1: The noisy side-scan sonar image to be processed (noise map) );
[0085] Input 2: Preliminary recovered side-scan sonar image (recovered side-scan sonar image) ).
[0086] 2) Encoder feature extraction (dual-branch parallel):
[0087] Branch 1 (Noise graph) The features of the noise map are extracted sequentially through Self-Attention module 1 → module 2 → module 3.
[0088] Branch 2 (Recovering Side Scan Sonar Images) The features of the restored image are extracted sequentially through Self-Attention module 1 → module 2 → module 3.
[0089] 3) Bottleneck feature fusion: The dual-branch features output by the encoder are first input into intermediate block 2, and then into intermediate block 1 to complete feature compression, interaction and fusion.
[0090] 4) Decoder feature reconstruction: The fused features of the bottleneck output are reconstructed sequentially through Self-Attention module 3 → module 2 → module 1 (symmetric to the number of encoder layers).
[0091] 5) Noise Prediction and Denoising Output: Decoder output predicted noise map (i.e., the total noise of the above embodiments) ).
[0092] Step S3: Based on the current diffusion state, the total noise is separated into structural noise and random noise and then fused to obtain optimized noise.
[0093] The predicted total noise With the current diffusion state The common input structure and random noise separator are processed separately through the separator's two-branch network: Branch 1 (structured noise branch) captures the spatial correlation features of sonar speckle noise and outputs structured noise. Branch 2 (random noise branch) extracts random disturbance components and outputs random noise. Both types of noise are 256×256 pixels in size. Then, structural noise weights are set. Random noise weights The two types of noise are weighted and fused to obtain the optimized noise prediction result. The calculation formula is: This enables targeted estimation of sonar noise.
[0094] Table 1 shows an example design of a structured random noise separator, which includes an input layer, a multi-scale feature extraction layer, a cross-scale attention fusion layer, a bi-branch refined modeling layer, a feature reconstruction layer, and an output optimization layer.
[0095]
[0096] Table 1
[0097] 1) Input layer: Feature concatenation and enhancement.
[0098] Input components: ① Total noise predicted by the diffusion model (256×256×1); ② Noisy image at the corresponding time step (256×256×1); ③ Acoustic feature map of sonar image (from 1×1 convolution) Extracted from the image, containing information such as texture direction and grayscale distribution (256×256×64).
[0099] Processing flow: The three data points are concatenated along the channel dimension (256×256×66), and feature enhancement is performed using "BatchNorm+GELU activation". The enhanced input feature map is then output. .
[0100] 2) Multi-scale feature extraction layer: 4-level MSCB modules, each MSCB module contains a "parallel convolution + feature fusion" structure to adapt to noise features at different scales:
[0101] Level 1 (Large Scale): 3×3 convolution (stride 1) + 5×5 convolution (stride 1) to extract large-scale continuous features of speckle noise;
[0102] Level 2 (Medium Scale): 3×3 convolution (stride 2) + dilated convolution (dilation=2) to capture medium-scale noise clump features;
[0103] Level 3 (small scale): 1×1 convolution (stride 2) + 3×3 convolution (dilation=4) to extract small-scale speckle particle features;
[0104] Level 4 (microscale): 1×1 convolution (stride 2) + point convolution, capturing sparse microscopic features of random noise.
[0105] Output: Level 4 feature map (256×256×128) (128×128×256) (64×64×512) (32×32×1024).
[0106] 3) Cross-scale attention fusion layer: Cross-Attention module.
[0107] Core logic: using the largest scale features As the "baseline feature", a cross-attention mechanism is used to... / / High-dimensional feature mapping to The scale is used to achieve multi-scale information complementarity; the calculation process is as follows:
[0108] ① To / / Upsampled to 256×256 size via transposed convolution;
[0109] ② Construct the attention matrix;
[0110] ③ Weighted fusion: , , Each feature is element-wise multiplied with its corresponding attention weight matrix to obtain weighted features at each scale; then all weighted features are summed and summed with the baseline feature. Element-level addition is performed, and the final output is a multi-scale fused feature map with a size of 256×256×1024.
[0111] 4) Dual-branch refined modeling layer.
[0112] (1) Structural noise branch (S-Branch): Focuses on spatial correlation.
[0113] Components: 4 layers of “Residual Convolutional Block (ResConv) + Spatial Attention Module (SAM)” + Total Variation Regularization Layer (TV).
[0114] Process: ① After the feature map is input, deep features are extracted using ResConv, and the SAM module strengthens the weights of spatially continuous noise regions (such as speckle clusters); ② TV regularization is inserted every two layers to constrain the smoothness and continuity of structural noise; ③ Preliminary structural noise features are output. (256×256×512).
[0115] (2) Random noise branch (R-Branch): Focus on sparsity.
[0116] Components: 4 layers of "Depthly Separable Convolution (DSConv) + Channel Attention Module (CAM)" + L1 sparse constraint layer.
[0117] Process: ① Use DSConv to reduce the number of parameters, while the CAM module suppresses invalid channels and focuses on the sparse distribution region of random noise; ② Insert L1 constraints every two layers to promote the sparsity of noise features; ③ Output preliminary random noise features. (256×256×512).
[0118] 5) Feature reconstruction layer: upsampling and residual fusion.
[0119] Components: 2-stage transposed convolution (upsampled to 256×256) + residual fusion block (fusion) and (edge features).
[0120] Process: ① For and The features are obtained by restoring them to the input size through transposed convolution. and ① To ensure spatial alignment of noise features; ② Residual fusion block: = ③ Supplement the detailed information in the fusion features; ③ Output the separated structural noise (256×256×1) and random noise (256×256×1).
[0121] 6) Output optimization layer: noise characteristic constraints.
[0122] Structural noise constraint: This is achieved by fitting a layer with a Gamma distribution to ensure... The distribution conforms to the Gamma characteristics of sonar speckle noise.
[0123] Random noise constraint: This is achieved by fitting a Gaussian distribution layer to ensure... It follows a standard normal distribution.
[0124] Final output: After distribution constraints and .
[0125] Step S4: Based on the optimized noise and the current diffusion state, calculate the preliminary estimated image corresponding to the current iteration time step.
[0126] Based on optimized noise prediction decay coefficient at the current time step Calculate the preliminary high-resolution image estimate corresponding to the current inverse diffusion step. The calculation formula is: ,in For the initial image estimation after denoising, the size is kept at 256×256 pixels.
[0127] Step S5: Generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, generate the diffusion state for the next iteration time step.
[0128] 1) Calculate the sampling standard deviation.
[0129] Based on the decay coefficient corresponding to the current time step t The decay coefficient corresponding to the preceding time step s Calculate the sampling standard deviation of the current reverse diffusion step. The calculation formula is: It is used to control the noise intensity during the reverse diffusion update process.
[0130] 2) Update the preceding diffusion state.
[0131] Generate sampling noise that follows a standard normal distribution. (Size 256×256 pixels), the diffusion state corresponding to the previous time step s is updated by "preliminary image weighting + optimized noise weighting + sampling noise weighting". The calculation formula is: ,Will As the current diffusion state for the next iteration .
[0132] Step S6: Based on the diffusion state of the next iteration time step, re-execute the process of inputting the current diffusion state and the current iteration time step into the diffusion generation model until the inverse diffusion iteration process ends, and obtain the super-resolution image.
[0133] When the number of iterations decreases from N=50 to n=1, the reverse diffusion iteration loop ends, and the diffusion state obtained at this time has completed the entire process of noise reduction and detail optimization.
[0134] The high-resolution preliminary estimate obtained from the final iteration As super-resolution side-scan sonar images Output, The resolution is 256×256 pixels (4× resolution improvement is achieved through latent space super-resolution using a diffusion model).
[0135] The following describes in detail the training process of the above diffusion generation model and the structure and random noise separator, including the following steps P1 to P5.
[0136] Step P1: Data preprocessing for acoustic degradation adaptation of sonar images.
[0137] Figure 3 The data preprocessing flowchart for acoustic degradation adaptation of sonar images includes stages such as HR side-scan sonar image input, acoustic propagation attenuation simulation, scattering perturbation blurring, speckle multiplicative noise superposition, image downsampling, and output LR side-scan sonar images.
[0138] HR side-scan sonar image input: High-resolution side-scan sonar images containing seabed topography and underwater targets (such as shipwrecks and pipelines) are selected as the raw input. These images are uniformly cropped to a 256×256 pixel effective detection area, removing pure black pixels from sonar blind zones. The image grayscale values are normalized to the [0,1] range and converted to tensor format to ensure consistency and data adaptability in subsequent degradation operations, laying the foundation for simulating the real degradation process.
[0139] Acoustic propagation attenuation simulation: Based on the spherical diffusion attenuation law, pixel coordinates are mapped to actual propagation distances, and an exponential attenuation model is used to calculate the distance-related attenuation coefficient. Attenuation parameters are set according to the characteristics of seawater medium, with a small attenuation coefficient in the near-field region and an increasing coefficient in the far-field region with the detection distance. Gray-level attenuation is applied to the normalized image pixel by pixel to simulate the sonar imaging characteristics of "clear near-field and blurred far-field".
[0140] Scattering Perturbation Blur: To address the scattering effects of underwater suspended particles and turbulent currents, a direction-dependent blur kernel is generated—an isotropic Gaussian blur kernel is used when suspended particles dominate, and an anisotropic Gaussian blur kernel is used when turbulence dominates. The kernel size, standard deviation, and rotation angle are set, and a convolution operation is performed with the attenuated image to simulate stretching blur along a specific direction, closely matching the real scattering perturbation patterns.
[0141] Speckle multiplicative noise superposition: A Speckle noise map is generated based on the Gamma distribution. The shape and scale parameters are adjusted according to the marine environment to correlate the noise intensity with the image grayscale value. The noise map is multiplied pixel by pixel with the blurred image, and after superposition, grayscale values exceeding [0,1] are cropped to generate a sonar image containing granular inherent noise, thus restoring the noise characteristics of the sonar equipment imaging.
[0142] Image downsampling: Based on the requirements of the super-resolution task (2×, 4×, 8×), bicubic interpolation is used to downsample the noisy image. While maintaining the acoustic texture directionality, the image resolution is reduced through interpolation to generate a low-resolution image, simulating the physical resolution limitations of sonar equipment.
[0143] Output LR side-scan sonar images: Map the downsampled image grayscale values back to the [0,255] range and convert them to 8-bit grayscale image format. After verifying visual features and statistical properties to ensure that the image has sonar-specific degradation features, output LR images and form training pairs with the original HR images to provide truly suitable training data for the diffusion generation model.
[0144] Step P2: Training objectives and data preparation.
[0145] This training process aims to simultaneously optimize two core models: the diffusion generation model (…). The training data consists of paired low-resolution side-scan sonar images (LR) and high-resolution ground truth images (HR). The core objective is to enable the diffusion model to learn the mapping from noisy, low-quality images to high-resolution images through a joint training framework, and to enable the noise separation network to learn to decompose the predicted noise into structural noise and random noise with different spatial properties.
[0146] Step P3: Forward diffusion and noise injection.
[0147] During training, the forward noise addition stage of the diffusion process is simulated starting with a high-resolution ground truth image (HR). Specifically, a time step t is randomly sampled, and Gaussian noise is added according to a pre-defined noise schedule. The image is injected proportionally into HR to generate a noisy image in the intermediate state t. The key to this process is that the model needs to learn from arbitrary noise levels. First, predict the original noise that has been added. .
[0148] Step P4: Noise prediction, separation, and multi-objective loss optimization.
[0149] This is the core part of the training process: Diffusion Generative Model With noisy images Using time step t and low-resolution conditional image LR as input, noise is initially predicted. Subsequently, the noise separation network NSN receives this predicted noise and It is decomposed into two independent components: structural noise that captures spatial patterns such as speckle and stripes. and random noise representing random disturbances. These two components are fused using time-step adaptive weighting to obtain an enhanced noise estimate. .
[0150] Training loss consists of four parts:
[0151] Basic diffusion loss: Calculating the enhanced noise estimate Compared with actual injected noise The mean square error ensures the correctness of the denoising direction;
[0152] Structural noise constraints: for Constraints such as total variational regularization are applied to encourage it to maintain a smooth and continuous spatial structure;
[0153] Random noise constraints: Imposing sparsity constraints (such as the L1 norm) can promote a sparse distribution.
[0154] Perceptual consistency loss: Using a pre-trained VGG network, compare the initial clean images generated based on predicted noise. The distance between the generated image and the real high-resolution image (HR) in the deep feature space ensures the visual quality of the generated image.
[0155] Step P5: Parameter update and model convergence.
[0156] The above four losses are combined into a total loss function according to preset weights. The backpropagation algorithm is used to simultaneously calculate the total loss on the diffusion model parameters. Noise separation network parameters The gradients are calculated. The parameters of both models are updated synchronously using optimizers such as Adam. Through numerous iterations until the models converge, they gradually learn to accurately separate and remove complex noise under conditional guidance, ultimately generating super-resolution sonar images that maintain high fidelity and are rich in realistic texture.
[0157] In summary, this invention addresses the problem that traditional interpolation methods cannot incorporate the acoustic degradation characteristics of side-scan sonar images. It focuses on the matching between training data and sonar imaging patterns, and constructs LR-HR image pairs that conform to real-world scenarios by simulating energy attenuation and signal scattering processes in the underwater acoustic propagation link. At the same time, it embeds an attention module in the UNet backbone network to solve the problems of blurred edges, loss of detail, and noise amplification in the super-resolution results.
[0158] This invention addresses the problems of limited receptive field and "oversmoothing" caused by pixel-level loss in CNN methods. It focuses on balancing "detail recovery" and "global smoothness" and uses a weighted optimization of perceptual loss (VGG19 deep features) and noise loss (prediction-real noise L2 error) to avoid the loss of key acoustic textures and improve the visual perception quality and global texture consistency of super-resolution images.
[0159] This invention addresses the spatial structural characteristics of speckle noise in sonar images by innovatively decomposing the noise into two independent components: structural noise and random noise. By modeling spatially correlated speckle patterns and completely random perturbation components through a dual-branch approach, targeted suppression is achieved during the inverse diffusion process. This avoids the excessive smoothing of texture details common in traditional methods, significantly improving the visual quality and target recognition of the reconstructed image.
[0160] Based on the same concept, this invention also discloses a sonar image super-resolution system based on a diffusion generation model, such as... Figure 4 As shown, the system includes: an initial expansion state construction module, a noise prediction module, a noise optimization module, a preliminary estimation image generation module, a diffusion state update module, and a super-resolution image output module.
[0161] The initial expansion state construction module is used to generate the initial diffusion state based on the input image and Gaussian noise that follows a standard normal distribution.
[0162] The noise prediction module is used to input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state.
[0163] The noise optimization module is used to combine the current diffusion state, separate the total noise into structural noise and random noise, and then fuse them to obtain optimized noise.
[0164] The preliminary estimated image generation module is used to calculate the preliminary estimated image corresponding to the current iteration time step based on the optimized noise and the current diffusion state.
[0165] The diffusion state update module is used to generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, to generate the diffusion state for the next iteration time step.
[0166] The super-resolution image output module is used to re-execute the diffusion generation model by inputting the current diffusion state and the current iteration time step based on the diffusion state of the next iteration time step, until the inverse diffusion iteration process ends, and obtain the super-resolution image.
[0167] Based on the same concept, this invention also discloses a computer device, which may be a terminal, a laptop computer, a desktop computer, a server, a computer cluster, or other types of computer devices. For example... Figure 5 As shown, the computer device may include at least one processor and memory. The processor can execute instructions stored in the memory. The processor is communicatively connected to the memory via a data bus. In addition to the memory, the processor can also be communicatively connected to input devices, output devices, and communication devices via the data bus.
[0168] The processor can be any conventional processor. Processors may include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.
[0169] Memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0170] In this embodiment of the invention, an executable instruction is stored in a memory. The processor can read the executable instruction from the memory and execute the instruction to implement all or part of the steps of the method of the invention.
[0171] Based on the same concept, the present invention also discloses a computer-readable storage medium including a computer program product or storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in the exemplary embodiments above.
[0172] Computer program products can be written in any combination of one or more programming languages to perform the operations of the embodiments of this application. These programming languages include object-oriented programming languages such as Java and C++, as well as conventional procedural programming languages such as C or similar languages, and scripting languages (e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0173] Computer-readable storage media can take the form of any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires; electrically erasable programmable read-only memory (EEPROM); erasable programmable read-only memory (EPROM); programmable read-only memory (PROM); read-only memory (ROM); magnetic storage; flash memory; magnetic disk or optical disk; or any suitable combination thereof.
[0174] The above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Those skilled in the art can modify or make equivalent substitutions to the above technical solutions based on the concept of the present invention, and such modifications or equivalent substitutions should all be covered within the protection scope of the present invention. The protection scope of the present invention is defined by the claims.
Claims
1. A sonar image super-resolution method based on a diffusion generation model, characterized in that, The method includes: The initial diffusion state is generated based on the input image and Gaussian noise that follows a standard normal distribution. Input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state; Based on the current diffusion state, the total noise is separated into structural noise and random noise, and then fused to obtain optimized noise. This separation is achieved using a structural and random noise separator. The structural and random noise separator includes an input layer, a multi-scale feature extraction layer, a cross-scale attention fusion layer, a bi-branch refined modeling layer, a feature reconstruction layer, and an output optimization layer. The input layer fuses the features of the current diffusion state and the total noise to obtain a first fused feature. The multi-scale feature extraction layer extracts noise features from the first fused feature at different scales. The cross-scale attention fusion layer... The attention fusion layer is used to weightedly fuse noise features at different scales through an attention mechanism to obtain a second fused feature; the bi-branch refined modeling layer is used to focus on the spatial correlation and sparsity of the second fused feature to obtain preliminary structural noise features and preliminary random noise features; the feature reconstruction layer is used to upsample and fuse the preliminary structural noise features and preliminary random noise features to obtain structural noise and random noise; the output optimization layer is used to ensure that the output structural noise and random noise meet the corresponding specific distribution constraints, including: Gamma distribution constraint or standard normal distribution; Based on the optimized noise and the current diffusion state, calculate the preliminary estimated image corresponding to the current iteration time step; Generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, generate the diffusion state for the next iteration time step; Based on the diffusion state of the next iteration time step, the process of inputting the current diffusion state and the current iteration time step into the diffusion generation model is repeated until the inverse diffusion iteration process ends, resulting in a super-resolution image.
2. The method according to claim 1, characterized in that, The process of generating the input image includes: Acquire raw sonar images; The original sonar image is denoised and its details are restored to obtain the input image.
3. The method according to claim 1, characterized in that, The process of training the diffusion generation model and the structure-random noise separator includes: Construct image pairs consisting of low-resolution images and high-resolution ground truth images; A random iteration time step is sampled, and according to a preset noise schedule, a set noise is injected into the high-resolution ground truth image in proportion to generate a noisy image. Using a noisy image, an iterative time step, and a low-resolution image as input to the diffusion generation model, and combining the structure and random noise separator, the total noise, structural noise, random noise, optimization noise, and current diffusion state of the noisy image are obtained. The basic diffusion loss is obtained based on the mean square error between the set noise and the optimized noise; By applying total variational regularization constraints to the structural noise, structural noise constraints are obtained. Random noise constraints are obtained by imposing sparsity constraints on random noise; Deep features of the preliminary estimated image and the high-resolution ground truth image corresponding to the current diffusion state are extracted by a pre-trained VGG19 network. The distance of the deep features in the feature space is calculated to obtain the perceptual consistency loss. The total training loss is obtained based on the basic diffusion loss, structural noise constraint, random noise constraint, and perceptual consistency loss. Backpropagation is performed based on the total training loss to update the parameters of the diffusion generation model and the structure and random noise separator.
4. The method according to claim 3, characterized in that, The process of generating the low-resolution image includes: Preprocessing is performed on high-resolution ground truth images, including: cropping, removing pure black pixels, image grayscale normalization, and tensor format conversion; Based on the set acoustic propagation attenuation parameters, the preprocessed image is attenuated; Based on the underwater suspended particles and turbulent scattering contained in the high-resolution ground truth image, a Gaussian blur kernel is constructed, and the attenuated image is convolved based on the Gaussian blur kernel to obtain the blurred image. A Speckle noise map is generated based on the Gamma distribution, and the Speckle noise map is multiplied pixel by pixel with the blurred image to generate an image containing granular inherent noise. Downsampling of images containing granular inherent noise; The grayscale values of the downsampled image are converted to a grayscale image with a set number of bits to obtain a low-resolution image.
5. The method according to claim 1, characterized in that, Based on the optimized noise and the current diffusion state, the preliminary estimated image corresponding to the current iteration time step is calculated, including: Generate a scaling factor based on the decay coefficient of the current iteration time step; Calculate the weighted sum of the optimization noise and the current diffusion state based on the decay coefficient of the current iteration time step; Based on the scaling factor and the weighted sum, a preliminary estimated image corresponding to the current reverse diffusion step is obtained.
6. The method according to claim 1, characterized in that, Based on the preliminary image estimation, noise optimization, and sampling noise, the diffusion state for the next iteration time step is generated, including: Calculate the sampling standard deviation of the current reverse diffusion time step based on the decay coefficient of the current iteration time step and the decay coefficient of the previous reverse diffusion time step; The first weight is obtained based on the decay coefficient of the preceding reverse diffusion time step; The second weight is obtained based on the decay coefficient of the preceding reverse diffusion time step and the sampling standard deviation; The third weight is obtained based on the sampling standard deviation; The diffusion state of the preliminary estimated image, optimization noise, and sampling noise are calculated by weighting the first weight, the second weight, and the third weight, to obtain the diffusion state of the next iteration time step.
7. A sonar image super-resolution system based on a diffusion generation model, characterized in that, The system includes: An initial expansion state construction module is used to generate an initial diffusion state based on the input image and Gaussian noise that follows a standard normal distribution; The noise prediction module is used to input the current diffusion state and the current iteration time step into the diffusion generation model to obtain the total noise contained in the image under the current diffusion state; A noise optimization module is used to separate the total noise into structural noise and random noise based on the current diffusion state, and then fuse them to obtain optimized noise. Specifically, a structural and random noise separator is used to separate the total noise into structural noise and random noise based on the current diffusion state. The structural and random noise separator includes an input layer, a multi-scale feature extraction layer, a cross-scale attention fusion layer, a bi-branch refined modeling layer, a feature reconstruction layer, and an output optimization layer. The input layer is used to fuse the features of the current diffusion state and the total noise to obtain a first fused feature. The multi-scale feature extraction layer is used to extract noise features of the first fused feature at different scales. The cross-scale attention fusion layer is used to weightedly fuse noise features at different scales through an attention mechanism to obtain a second fused feature; the bi-branch refined modeling layer is used to focus on the spatial correlation and sparsity of the second fused feature to obtain preliminary structural noise features and preliminary random noise features; the feature reconstruction layer is used to upsample and residually fuse the preliminary structural noise features and preliminary random noise features to obtain structural noise and random noise; the output optimization layer is used to ensure that the output structural noise and random noise meet the corresponding specific distribution constraints, including: Gamma distribution constraints or standard normal distribution; The preliminary estimated image generation module is used to calculate the preliminary estimated image corresponding to the current iteration time step based on the optimized noise and the current diffusion state; The diffusion state update module is used to generate sampling noise that follows a standard normal distribution, and based on the preliminary estimated image, optimized noise and sampling noise, to generate the diffusion state for the next iteration time step; The super-resolution image output module is used to re-execute the diffusion generation model by inputting the current diffusion state and the current iteration time step based on the diffusion state of the next iteration time step, until the inverse diffusion iteration process ends, and obtain the super-resolution image.
8. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the sonar image super-resolution method based on the diffusion generation model as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the sonar image super-resolution method based on a diffusion generation model as described in any one of claims 1-6.