Sonar image generation method and device based on diffusion model, equipment and medium

Abstract

The invention relates to a sonar image generation method and device based on a diffusion model, equipment and a medium, and the method comprises the steps: carrying out the multi-time noise addition of each sonar actual measurement image through a forward process based on the diffusion model, and obtaining a plurality of noise images with different noise degrees; a plurality of noise images generated by the same image and corresponding semantic guidance graphs are used as a group of training data, the training data are input into a sonar image denoising network constructed based on a reverse process of a diffusion model to train the sonar image denoising network, in the training process, under the guidance of the semantic guidance graphs, the noise images are denoised for multiple times, and the noise images are denoised for multiple times. Learning parameters in the sonar image denoising network are adjusted according to a denoising result predicted by the network and an error of a corresponding real noise image, and finally, randomly generated noise images and semantic images are input into the trained sonar image denoising network for multiple times of sampling. And generating a sonar image related to the semantic image. By adopting the method, the high-quality sonar image can be efficiently generated.

Description

Technical Field

[0001] The present application relates to the field of sonar imaging technology, and in particular to a sonar image generation method, device, equipment and medium based on a diffusion model. Background Art

[0002] Sound waves are the main method of underwater detection, and sonar images obtained through sonar equipment have become an important source of underwater data, and are widely used in tasks such as marine surveying and mapping, underwater target detection, and ship navigation. In recent years, the demand for sonar images for underwater detection has grown rapidly, but due to the high cost of equipment and the complex underwater environment, the existing sonar images are not only scarce in quantity but also of poor quality, which brings huge challenges to the interpretation and recognition of sonar images.

[0003] A better solution is to build a high-quality sonar image library through image generation technology as a data source to improve the sonar image recognition effect. Sonar image generation mainly generates virtual images similar to real sonar images by simulating the interaction between sonar detectors and seabed targets. This technology can automatically generate typical sonar images when the number of sonar images is limited. Traditional generation methods include geometric optical models, numerical simulations, and statistical models, but these methods have problems such as complex calculations, low model accuracy, and poor real-time performance. Summary of the invention

[0004] Based on this, it is necessary to provide a sonar image generation method, device, equipment and medium based on a diffusion model that can improve image generation efficiency and quality in response to the above technical problems.

[0005] The present application provides a sonar image generation method based on a diffusion model, the method comprising: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; Based on the reverse process of the diffusion model, a sonar image denoising network is constructed to guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

[0006] In one embodiment, the sonar image denoising network includes an encoder and a decoder; In the encoder, the noisy image is encoded into a feature map using a semantic diffusion coding stacked residual module and an attention layer; In the decoder, the semantic diffusion decoding stacked residual module is used to embed the semantic guidance map into the denoising decoding process, so that the decoder performs denoising decoding on the feature map under the guidance of the semantic guidance map, obtains the noise estimation of the current noise image, and denoises the input noise image according to the noise estimation.

[0007] In one embodiment, the sonar image denoising network further includes a fusion feature extraction layer connected between the encoder and the decoder; The fusion feature extraction layer fuses the feature map output by the encoder and the semantic guidance image, and then extracts features, and inputs the extracted fusion features into the decoder.

[0008] In one embodiment, the encoder includes an input convolutional layer, a semantic diffusion coding stacked residual module, a downsampling layer, and an attention layer connected in sequence; The fusion feature extraction layer includes a semantic diffusion decoding stacked residual module and an attention layer connected in sequence; The decoder includes a semantic diffusion decoding stacked residual module, an attention layer, and an upsampling layer connected in sequence; An output convolutional layer is also connected after the decoder.

[0009] In one embodiment, when the sonar image denoising network is trained, the learnable parameters in the sonar image denoising network are adjusted by calculating the denoising loss function and the KL divergence loss function between the estimated distribution and the diffusion process posteriori until convergence, thereby obtaining the trained sonar image denoising network.

[0010] In one embodiment, before generating the noise image, each measured sonar image is processed by translation, rotation and flipping to enhance the training data set.

[0011] In one embodiment, when performing forward process noise addition on each of the sonar measured images, cosine noise addition is used for noise scheduling.

[0012] The present application also provides a sonar image generation device based on a diffusion model, the device comprising: A data acquisition module, used to acquire a plurality of sonar measured images and semantic guidance images corresponding to each of the sonar measured images; A training data set acquisition module is used to perform multiple noise additions on each of the sonar measured images based on the forward process of the diffusion model to obtain multiple noise images with different noise levels, and use the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image as a set of training data to obtain a training data set including multiple sets of training data; Sonar image denoising network training module, which is used to build a sonar image denoising network based on the reverse process of the diffusion model, and guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; The sonar image generation module is used to obtain a noise image and a semantic image randomly generated by using Gaussian noise, input the noise image and the semantic image into a trained sonar image denoising network for multiple sampling denoising, and generate a sonar image related to the semantic image.

[0013] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; Based on the reverse process of the diffusion model, a sonar image denoising network is constructed to guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

[0014] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; Based on the reverse process of the diffusion model, a sonar image denoising network is constructed to guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

[0015] The above-mentioned sonar image generation method, device, equipment and medium based on the diffusion model, through the forward process based on the diffusion model, each sonar measured image is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance map are used as a set of training data, which are input into the sonar image denoising network for training to obtain a trained sonar image denoising network. The sonar image denoising network is constructed based on the reverse process of the diffusion model. During the training process, under the guidance of the semantic guidance map, the noise image is denoised multiple times, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image. Finally, the randomly generated noise image and the semantic image are input into the trained sonar image denoising network for multiple sampling to generate a sonar image related to the semantic image. The method can efficiently generate high-quality sonar images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a sonar image generation method based on a diffusion model in one embodiment; Figure 2 A schematic diagram of using a semantic segmentation map to perform image denoising in one embodiment; Figure 3 A schematic diagram of the structure of a sonar image denoising network in one embodiment; Figure 4 FIG. 1 is a schematic diagram of the structure of SDEResblock and SDDResblock in one embodiment, wherein: Figure 4 (a) Schematic diagram of the structure of SDEResblock. Figure 4 (b) Schematic diagram showing the structure of SDDResblock; Figure 5 is a structural block diagram of a sonar image generation device based on a diffusion model in one embodiment; Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] In view of the problems in the prior art that the traditional sonar image generation method has complex calculations, low model accuracy and poor real-time performance, and the quality or diversity of images generated based on the generative adversarial network is not satisfactory, in this application, Figure 1 As shown, a sonar image generation method based on a diffusion model is provided, which specifically includes the following steps: Step S100, obtaining a plurality of sonar measured images and a semantic guidance map corresponding to each sonar measured image.

[0019] Step S110, based on the forward process of the diffusion model, each sonar measured image is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance map are used as a set of training data to obtain a training data set including multiple sets of training data.

[0020] Step S120, constructing a sonar image denoising network based on the reverse process of the diffusion model, Figure 1 A set of training data is input into the sonar image denoising network for training to obtain a trained sonar image denoising network. In the sonar image denoising network, the noisy image is denoised under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image.

[0021] Step S130, obtaining a noise image and a semantic image randomly generated by using Gaussian noise, inputting the noise image and the semantic image into a trained sonar image denoising network for multiple sampling denoising, and generating a sonar image related to the semantic image.

[0022] In this embodiment, a sonar image generation method based on a diffusion model is proposed, which can provide more data sources for subsequent tasks of target recognition based on sonar images. Due to the problem of less measured sonar image data, some existing methods for sonar image target recognition or detection based on neural networks are difficult to provide sufficient training data, which makes network training difficult. The method in this application first uses less measured sonar data to train a network that can generate high-quality sonar images based on noise images and semantic images, thereby effectively solving the above problem.

[0023] In step S100, a plurality of measured sonar images are obtained, and then any one of the methods including but not limited to traditional image processing and classification methods, deep learning methods, generative adversarial networks, and self-supervised learning methods is used to generate corresponding semantic images (semantic segmentation images) based on the measured sonar images, that is, semantic annotation is performed on different areas in the sonar images, such as seabed topography, shipwrecks, marine life, and any targets of interest.

[0024] Furthermore, in order to enrich the number of training samples in the training data set, the translation, rotation and flipping methods are used to process each sonar measured image to enhance the training data set, so that better training effects can be achieved when the neural network is trained later.

[0025] In this embodiment, according to the application scenario of the final sonar image denoising network, that is, the specific target type to be subsequently identified for the sonar image, the image target type in the training data set is adjusted to adapt to the subsequent task.

[0026] In this embodiment, a sonar image denoising network is constructed based on the diffusion model. The implementation process of the diffusion model is a parameterized Markov chain, which is divided into two processes, one is the forward process and the other is the reverse process. Figure 2 Shown is a schematic diagram of the forward and backward processes in the diffusion model.

[0027] like Figure 2 As shown, from right to left is the forward process, also known as the diffusion process, which is the process of gradually adding Gaussian noise to a real image until the image becomes pure Gaussian noise. The forward process can be understood as a Markov chain, and the expression is as follows: ; In the above formula, For images at different times, For real images, is pure Gaussian noise, As a hyperparameter, Gaussian noise is added to the time t-1 at each time t, and each step can be adjusted To control the noise, given Under the premise of The probability distribution of generating images at this moment: ; In the above formula, .

[0028] The reverse process is the process of denoising and generating, which is also a Markov chain like the forward process. The core is how to get The reverse process , this process cannot be directly calculated, which requires the network to learn to fit this distribution, so a neural network with parameters is constructed to calculate . Assuming that the conditional distribution of the reverse process is also Gaussian, then It can be expressed as: ; In the diffusion model, the variance is given directly as follows: ; At this point, all we need to find is the mean.

[0029] like Known, then Substituting into the Bayesian formula, we can get the following formula: ; Since they all obey the standard normal distribution, we have: ; After expansion and simplification, we can get: ; The mean expression can be obtained from the normal distribution formula: ; In step S110, each sonar measured image is denoised multiple times based on the forward process in the diffusion model, so that multiple noise images with different noise levels can be obtained for each sonar measured image. The forward process in the diffusion model is to add noise to the sonar measured image step by step for multiple times, so in the process of adding noise to the sonar measured image into a pure noise image, multiple noise images with different noise levels will be obtained according to the noise level each time, and these different noise images are used as label images for subsequent training of the sonar image denoising network.

[0030] In step S120, a sonar image denoising network is constructed based on the reverse process of the diffusion model, that is, the denoising process, and the corresponding semantic guidance graph is introduced. Under the guidance of the semantic guidance graph, the noise image is iteratively denoised to restore the sonar image. The process can be referred to Figure 2 shown.

[0031] In this embodiment, the process of sonar image generation in the sonar image denoising network is a parameterized Markov chain. In each step, a denoising network based on a semantic guidance graph estimates the noise from the input noisy image. Based on the estimated noise, an image with less noise is generated through the posterior probability formula. Through iteration, the denoising network gradually generates semantically related content and finally generates a real image.

[0032] In this embodiment, the structure of the sonar image denoising network is as follows: Figure 3 As shown in the figure, it is a U-Net-based network. Different from the previous conditional diffusion model, the sonar image denoising network processes the semantic label map and the noisy image separately. The noisy image is input into the encoder network. At the same time, in order to make full use of the semantic information, the semantic label map is injected into the decoder of the denoising network using multi-layer spatial adaptive normalization operations. Finally, the sampled Gaussian noise is converted into a real image through an iterative denoising process.

[0033] Specifically, the sonar image denoising network includes an encoder and a decoder. In the encoder, the noisy image is encoded into a feature map using a semantic diffusion encoding stacked residual module and an attention layer. In the decoder, the semantic diffusion decoding stacked residual module is used to embed the semantic guidance map mapping into the denoising decoding process, so that the decoder performs denoising decoding on the feature map under the guidance of the semantic guidance map, obtains a noise estimate of the current noisy image, and performs denoising on the input noisy image according to the noise estimate.

[0034] In this embodiment, the encoder encodes the features of the noisy image based on the semantic diffusion coding stacked residual module (SDE Res-block) and the attention block. The specific structure of the semantic diffusion coding stacked residual module is as follows: Figure 4 As shown in (a), the activation function used is the Sigmoid activation function. After the image passes through the activation layer, it can be expressed by the following formula: .

[0035] SDE Resb-lock estimates different moments by scaling and moving the intermediate activation layers The noise, The noise weight at time is , the deviation is , then we have the following expression: ; In the above formula, are the input and output features respectively.

[0036] Furthermore, the attention block refers to a self-attention layer with a skip connection, and the formula is as follows: ; ; ; In the above formula, and are the input and output of the attention block. , , and They refer to the 1×1 convolution in the attention block respectively. and is an index into the spatial dimension, ranging from 1 to .

[0037] In this embodiment, the decoder embeds the semantic label map into the decoder of the denoising network in a multi-layer spatial adaptive manner, and the decoder includes a semantic diffusion decoding stacked residual module SDD Res-block and a self-attention block. The specific structure of the SDD Res-block is as follows: Figure 4 (b) As shown. Different from the SDE Res block, spatial adaptive normalization is introduced instead of group normalization. Spatial adaptive normalization introduces semantic label mapping into the denoising process by adjusting features in a spatially adaptive and learnable transformation. Its formula is as follows: ; In the above formula, and are the input and output features of spatial adaptive normalization, refers to parameter-free group normalization, , are the spatial adaptation weights and biases learned from the semantic layout, respectively.

[0038] Further, such as Figure 3 As shown, the sonar image denoising network also includes a fusion feature extraction layer connected between the encoder and the decoder. The fusion feature extraction layer fuses the feature map output by the encoder and the semantic guidance image, and then extracts features, and inputs the extracted fusion features into the decoder.

[0039] Specifically, the encoder includes an input convolution layer (Input Conv), a semantic diffusion coding stacked residual module (SDD Res-block) x2, a downsampling layer (Down Sample), a semantic diffusion coding stacked residual module (SDDRes-block) x2, and an attention layer (AttentionBlock) connected in sequence.

[0040] Specifically, the fusion feature extraction layer includes a semantic diffusion decoding stacked residual module (SDD Res-block), an attention layer (AttentionBlock) and a semantic diffusion decoding stacked residual module (SDD Res-block) connected in sequence.

[0041] Specifically, the decoder includes a semantic diffusion decoding stacked residual module (SDD Res-block) x3, an attention layer (AttentionBlock), an upsampling layer (Up Sample) and a semantic diffusion decoding stacked residual module (SDDRes-block) x3 connected in sequence.

[0042] Specifically, there is an output convolution layer (Output Conv) connected after the decoder.

[0043] In this embodiment, when the sonar image denoising network is trained, the learnable parameters in the sonar image denoising network are adjusted by calculating the denoising loss function and the KL divergence loss function between the estimated distribution and the diffusion process posteriori until convergence, thereby obtaining the trained sonar image denoising network.

[0044] Specifically, when calculating the denoising loss function, a reference output image is given (i.e., the denoised image after denoising according to the predicted denoising degree) and a random time step , the reference image with noise added (i.e. the noisy images generated when constructing the training dataset) are produced as follows: ; In the above formula, is noise randomly sampled from a standard Gaussian distribution, and is the removable noise estimate predicted by the sonar image denoising network. is a time step Noise scheduler. In semantic layout and time step Under the guidance of To train the model to reconstruct the reference image , the formula is as follows: ; Then train the network to predict the variance To improve the log-likelihood of the generated image. The conditional diffusion model additionally outputs the interpolation coefficients for each dimension , and convert the output to variance as follows: ; In the above formula, and are the upper and lower limits of the variance respectively.

[0045] Specifically, the KL divergence loss function directly optimizes the estimated distribution and diffusion process posterior The KL divergence between is as follows: ; Furthermore, the weighted sum of the denoising loss function and the KL divergence loss function is used as the overall loss function to train the sonar image denoising network, where the overall loss function is expressed as: ; In the above formula, is a trade-off parameter that balances the two loss functions.

[0046] In this embodiment, in the process of adding noise to the sonar measured image, a cosine noise addition method is used for noise scheduling.

[0047] Specifically, in the forward process of the diffusion model, noise needs to be continuously added to the image, and the rate of adding noise is controlled by noise scheduling. Linear noise scheduling is used in the diffusion model, and the expression is as follows: ; The image is denoised by linear denoising. Generally, the denoising speed is relatively fast, and the time it takes for the image to become pure noise is shortened. The reverse process is a process of continuously denoising by estimating the noise at each step in the forward process. In the later stage of denoising, the image basically tends to be pure noise, and the contribution to the estimated noise in the reverse denoising process is not large. If the denoising speed is too fast, it is not conducive to the estimation of noise in the reverse process. For images with high resolution, linear denoising can obtain better experimental results. However, for sonar images, the performance is not the best when the resolution is generally low.

[0048] Therefore, in this embodiment, according to the characteristics of sonar images, a cosine noise addition method is proposed to slow down the noise addition rate and enhance the contribution of the previous process in the noise addition process. The formula of cosine noise addition is as follows: ; In the above formula, preferably, .

[0049] In step S130, the trained sonar image denoising network is used to perform multiple samplings based on the noise image and the semantic image to generate a sonar image, wherein the noise image is a pure noise image randomly generated by a Gaussian noise kernel.

[0050] In this paper, the effectiveness of this method is also demonstrated through experiments. In the experiment, the Seabed Objects-KLSG dataset is used. In order to better evaluate the effectiveness of using image generation to expand the target recognition dataset, it is necessary to ensure that the test set of image generation will be the same as the test set of target detection later. Therefore, the two categories of the original image are randomly divided in a ratio of 7:3, resulting in 314 training sets and 133 test sets.

[0051] At the same time, since the original sonar image data is small and the number of each category varies greatly, in order to prevent overfitting when using deep learning algorithms to generate sonar images, the training set is enhanced by translation, rotation, and flipping methods to obtain 3160 sonar images for training the image generation algorithm.

[0052] At the same time, the SPADE algorithm was selected for comparison with the method in this paper. The training set after data enhancement was input into the SPADE algorithm and the method in this paper for training respectively to obtain the training results.

[0053] The experimental results show that both SPADE based on generative adversarial networks and our method (Ours) can achieve controllable generation of sonar images through semantic images. The generated images have diversity while ensuring realistic sonar image style. Overall, our method has better quality images than the SPADE algorithm based on generative adversarial networks, and the edges of the generated images are clearer.

[0054] At the same time, in order to more objectively evaluate the quality of the test synthetic images, the FID indicator is used to evaluate the quality of the generated images in the test set, where the lower the FID, the better the quality of the synthetic image. Table 1 is an evaluation table of the quality of sonar image generation.

[0055] Table 1 Sonar image generation quality evaluation table

[0056] From the above results, it can be seen that compared with the existing mainstream methods, the method proposed in this paper can effectively generate sonar images with higher quality and more forms, and can provide more data sources for improving the sonar image recognition effect.

[0057] The above-mentioned sonar image generation method based on the diffusion model performs multiple noise additions on each sonar measured image through the forward process based on the diffusion model to obtain multiple noise images with different noise levels, and uses the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance map as a set of training data, which are input into the sonar image denoising network for training to obtain a trained sonar image denoising network. The sonar image denoising network is constructed based on the reverse process of the diffusion model. During the training process, under the guidance of the semantic guidance map, the noise image is denoised multiple times, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image. Finally, the randomly generated noise image and the semantic image are input into the trained sonar image denoising network for multiple sampling to generate a sonar image related to the semantic image. The method can efficiently generate high-quality sonar images and provide more data sources for improving the sonar image recognition effect.

[0058] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0059] In one embodiment, Figure 5 As shown, a sonar image generation device based on a diffusion model is provided, comprising: a data acquisition module 200, a training data set acquisition module 210, a sonar image denoising network training module 220 and a sonar image generation module 230, wherein: A data acquisition module 200 is used to acquire a plurality of sonar measured images and semantic guidance images corresponding to each of the sonar measured images; The training data set acquisition module 210 is used to perform multiple noise additions on each of the sonar measured images based on the forward process of the diffusion model to obtain multiple noise images with different noise levels, and use the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance map as a set of training data to obtain a training data set including multiple sets of training data; Sonar image denoising network training module 220 uses the reverse process based on the diffusion model to build a sonar image denoising network, which guides the semantic Figure 1A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; The sonar image generation module 230 is used to obtain a noise image and a semantic image randomly generated by using Gaussian noise, input the noise image and the semantic image into a trained sonar image denoising network for multiple sampling denoising, and generate a sonar image related to the semantic image.

[0060] The specific definition of the sonar image generation device based on the diffusion model can be found in the definition of the sonar image generation method based on the diffusion model above, which will not be repeated here. Each module in the above-mentioned sonar image generation device based on the diffusion model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0061] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sonar image generation method based on a diffusion model is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0062] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0063] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; Based on the reverse process of the diffusion model, a sonar image denoising network is constructed to guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

[0064] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; Based on the reverse process of the diffusion model, a sonar image denoising network is constructed to guide the semantic Figure 1 A set of training data is inputted into a sonar image denoising network for training to obtain a trained sonar image denoising network, wherein the noisy image is denoised in the sonar image denoising network under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

[0065] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0066] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A sonar image generation method based on a diffusion model, characterized in that: The method comprises: Obtaining multiple sonar measured images and semantic guidance images corresponding to each of the sonar measured images; Based on the forward process of the diffusion model, each of the sonar measured images is denoised multiple times to obtain multiple noise images with different noise levels, and the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image are used as a set of training data to obtain a training data set including multiple sets of training data; A sonar image denoising network is constructed based on the reverse process of the diffusion model, a set of training data of the semantic guidance graph is input into the sonar image denoising network for training to obtain a trained sonar image denoising network, in which the noisy image is denoised under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; A noise image and a semantic image randomly generated by using Gaussian noise are obtained, and the noise image and the semantic image are input into a trained sonar image denoising network for multiple sampling denoising to generate a sonar image related to the semantic image.

2. The sonar image generation method based on the diffusion model according to claim 1 is characterized in that: The sonar image denoising network includes an encoder and a decoder; In the encoder, the noisy image is encoded into a feature map using a semantic diffusion coding stacked residual module and an attention layer; In the decoder, the semantic diffusion decoding stacked residual module is used to embed the semantic guidance map into the denoising decoding process, so that the decoder performs denoising decoding on the feature map under the guidance of the semantic guidance map, obtains the noise estimation of the current noise image, and denoises the input noise image according to the noise estimation.

3. The sonar image generation method based on the diffusion model according to claim 2 is characterized in that: The sonar image denoising network also includes a fusion feature extraction layer connected between the encoder and the decoder; The fusion feature extraction layer fuses the feature map output by the encoder and the semantic guidance image, and then extracts features, and inputs the extracted fusion features into the decoder.

4. The sonar image generation method based on the diffusion model according to claim 3 is characterized in that: The encoder includes an input convolutional layer, a semantic diffusion coding stacked residual module, a downsampling layer, and an attention layer connected in sequence; The fusion feature extraction layer includes a semantic diffusion decoding stacked residual module and an attention layer connected in sequence; The decoder includes a semantic diffusion decoding stacked residual module, an attention layer, and an upsampling layer connected in sequence; An output convolutional layer is also connected after the decoder.

5. The sonar image generation method based on the diffusion model according to claim 4 is characterized in that: When the sonar image denoising network is trained, the learnable parameters in the sonar image denoising network are adjusted by calculating the denoising loss function and the KL divergence loss function between the estimated distribution and the diffusion process posteriori until convergence, thereby obtaining the trained sonar image denoising network.

6. The sonar image generation method based on the diffusion model according to claim 1, characterized in that: Before generating the noise image, each measured sonar image is processed by translation, rotation and flipping methods to enhance the training data set.

7. The sonar image generation method based on the diffusion model according to any one of claims 1 to 6, characterized in that: When the forward process of adding noise to each of the sonar measured images is performed, the cosine noise addition method is used for noise scheduling.

8. A sonar image generation device based on a diffusion model, characterized in that: The device comprises: A data acquisition module, used to acquire a plurality of sonar measured images and semantic guidance images corresponding to each of the sonar measured images; A training data set acquisition module is used to perform multiple noise additions on each of the sonar measured images based on the forward process of the diffusion model to obtain multiple noise images with different noise levels, and use the multiple noise images generated corresponding to the same sonar measured image and the corresponding semantic guidance image as a set of training data to obtain a training data set including multiple sets of training data; A sonar image denoising network training module is used to construct a sonar image denoising network based on the reverse process of the diffusion model, input a set of training data of the semantic guidance graph into the sonar image denoising network to train it to obtain a trained sonar image denoising network, in which the noisy image is denoised under the guidance of the semantic guidance graph, and the learnable parameters in the sonar image denoising network are adjusted according to the error between the denoising result predicted by the network and the corresponding real noise image; The sonar image generation module is used to obtain a noise image and a semantic image randomly generated by using Gaussian noise, input the noise image and the semantic image into a trained sonar image denoising network for multiple sampling denoising, and generate a sonar image related to the semantic image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images