SAR (Synthetic Aperture Radar) image colorization method and system, storage medium and equipment

Abstract

The invention relates to the technical field of image processing, and discloses an SAR image colorization method and system, a storage medium and equipment, and the method comprises the steps: obtaining an SAR image, and extracting the spatial features and frequency features of the SAR image; the spatial features and the frequency features are input into a conditional diffusion model, the model comprises a spatial frequency information interaction network, and forward noise addition and reverse noise reduction are performed on the SAR image through the spatial frequency information interaction network; in the forward noise adding process, the spatial frequency information interaction network predicts and outputs noise to be added of each time sequence according to the spatial features and the frequency features; sequentially adding the noise to be added into the SAR image according to the sequence of the time sequence to obtain a pure noise image; in the reverse denoising process, the spatial frequency information interaction network predicts and outputs noise to be removed of each time sequence according to the pure noise image; and removing the noise to be removed from the pure noise image according to the sequence of the time sequence to obtain a color image. According to the invention, the quality of the SAR image colorization result can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a SAR image colorization method, system, storage medium and device. Background Art

[0002] As an advanced imaging technology, synthetic aperture radar (SAR) can stably obtain surface images under various complex weather and lighting conditions due to its excellent penetration and all-weather working ability. This unique advantage makes SAR images an indispensable source of information in earth observation fields such as remote sensing, geological exploration, environmental monitoring, disaster assessment and resource exploration. SAR images not only provide detailed spatial distribution information, but also contain rich radiation data, which are extremely valuable for accurately identifying surface features and understanding surface change processes.

[0003] Although SAR images play a pivotal role in the field of earth observation, their inherent speckle noise and grayscale characteristics have become key factors restricting their application effects. The presence of speckle noise often masks important visual information in the image, reduces the clarity and interpretability of the image, and makes the identification and analysis of surface features difficult; at the same time, the grayscale characteristics of SAR images also limit the information depth and intuitiveness of the image, which is not conducive to professional analysts to quickly and accurately obtain the required information.

[0004] In view of the importance of SAR images in the field of earth observation and the challenges they face, it is particularly important to develop effective SAR image colorization technology, which can more accurately analyze image information and thus play a greater role in practical applications. Summary of the invention

[0005] Based on this, it is necessary to propose a colorization method for SAR images to address the above problems.

[0006] A method for colorizing a SAR image, the method comprising the following steps: Acquire a SAR image, and extract spatial features and frequency features of the SAR image; Inputting the spatial features and frequency features into a conditional diffusion model, wherein the conditional diffusion model includes a spatial frequency information interaction network, and performing forward denoising and reverse denoising on the SAR image through the spatial frequency information interaction network; In the forward noise addition process, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics; Adding the noise to be added to the SAR image in sequence according to the order of the time series to obtain a pure noise image; In the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed of each of the time series according to the pure noise image; The noise to be removed is removed from the pure noise image in the order of the time series to obtain a color image.

[0007] In the above solution, the extracting of the spatial features and frequency features of the SAR image specifically includes: Performing a learnable wavelet transform on the SAR image to obtain frequency features; The SAR image is downsampled to obtain spatial features.

[0008] In the above scheme, the spatial frequency information interaction network includes a convolutional layer, a two-layer downsampling processing module, a multi-scale spatial frequency interaction module MSFI, a two-layer upsampling processing module and a convolutional layer connected in sequence; the upsampling processing module is composed of a multi-scale spatial frequency interaction module MSFI and upsampling, and the downsampling processing module is composed of a multi-scale spatial frequency interaction module MSFI and downsampling.

[0009] In the above scheme, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics, specifically including: The spatial feature and the frequency feature are fused and input into a convolution layer to obtain a first output feature; Inputting the first output feature into a downsampling processing module of the first layer to obtain a second output feature; Inputting the second output feature into the downsampling processing module of the second layer to obtain a third output feature; Input the third output feature into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature; Inputting the fourth output feature into the upsampling processing module of the first layer to obtain a fifth output feature; Inputting the fifth output feature into the upsampling processing module of the second layer to obtain a sixth output feature; The sixth output feature is input into the convolution layer to obtain the noise to be added.

[0010] In the above scheme, the multi-scale spatial frequency interaction module MSFI includes two learnable wavelet transform layers, two learnable inverse wavelet transform layers, three convolutional layers and two spatial frequency feature interaction layers SFIB.

[0011] In the above solution, the step of inputting the third output feature into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature specifically includes: Inputting the third output feature into the learnable wavelet transform layer of the first layer and the convolution layer of the first layer respectively, obtaining corresponding first processing results and second processing results, wherein the first processing result includes a high-frequency component and a low-frequency component; Inputting the first processing result into the convolution layer of the second layer to obtain the first feature to be fused; Input the low-frequency component into the learnable wavelet transform layer of the second layer to obtain a secondary low-frequency component and input it into the convolution layer of the third layer to obtain a convolution result, and input the convolution result into the learnable wavelet transform layer of the first layer to obtain a second feature to be fused; The first feature to be fused and the second feature to be fused are input into the first spatial frequency feature interaction layer SFIB to obtain the interaction feature, and then the interaction feature is input into the learnable wavelet transform layer of the second layer to obtain the third feature to be fused; The third feature to be fused and the second processing result are input into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature.

[0012] In the above solution, the step of inputting the third feature to be fused and the second processing result into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature specifically includes: Input the third feature to be fused into the convolution layer to extract its corresponding spatial feature; Inputting the second processing result into a learnable inverse wavelet transform layer to obtain the corresponding frequency feature; Perform feature expansion on the spatial features and the frequency features to obtain corresponding first sequence features and second sequence features; Perform feature mapping and position encoding on the first sequence features and the second sequence features based on a multi-layer perceptron MPL to obtain corresponding first encoding features and second encoding features; Mapping the first coding feature and the second coding feature into a plurality of feature vectors respectively and obtaining vector features calculated by interaction of the first coding feature and the second coding feature with the frequency sequence feature; Mapping the vector features into a feature map; A fourth output feature of the interactive calculation is determined according to the feature graph.

[0013] The present application also proposes a SAR image colorization method system, the system comprising: an image acquisition unit, a feature extraction unit and an image processing unit; The image acquisition unit is used to acquire the SAR image; The feature extraction unit is used to extract the spatial features and frequency features of the input SAR image; The image processing unit includes a space-frequency information interaction network, a denoising module and a denoising module, which are used to perform forward denoising and reverse denoising on the SAR image, wherein: Spatial-frequency information interaction network: predicts and outputs the noise to be added and removed for each time series based on spatial and frequency characteristics; Noise adding module: according to the noise to be added output by the space-frequency information interaction network, it is added to the SAR image in the order of time series to obtain a pure noise image; Denoising module: According to the noise to be removed output by the space-frequency information interaction network, the noise is removed from the pure noise image in the order of the time series to obtain a color image.

[0014] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps: Acquire a SAR image, and extract spatial features and frequency features of the SAR image; Inputting the spatial features and frequency features into a conditional diffusion model, wherein the conditional diffusion model includes a spatial frequency information interaction network, and performing forward denoising and reverse denoising on the SAR image through the spatial frequency information interaction network; In the forward noise addition process, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics; Adding the noise to be added to the SAR image in sequence according to the order of the time series to obtain a pure noise image; In the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed of each of the time series according to the pure noise image; The noise to be removed is removed from the pure noise image in the order of the time series to obtain a color image.

[0015] The present application also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the following steps of the computer program: Acquire a SAR image, and extract spatial features and frequency features of the SAR image; Inputting the spatial features and frequency features into a conditional diffusion model, wherein the conditional diffusion model includes a spatial frequency information interaction network, and performing forward denoising and reverse denoising on the SAR image through the spatial frequency information interaction network; In the forward noise addition process, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics; Adding the noise to be added to the SAR image in sequence according to the order of the time series to obtain a pure noise image; In the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed of each of the time series according to the pure noise image; The noise to be removed is removed from the pure noise image in the order of the time series to obtain a color image.

[0016] The embodiment of the present invention has the following beneficial effects: firstly, a SAR image is acquired, and the spatial features and frequency features of the SAR image are extracted; the spatial features and frequency features are input into a conditional diffusion model, the conditional diffusion model includes a spatial frequency information interaction network, and the SAR image is forward denoised and reverse denoised through the spatial frequency information interaction network; in the forward denoising process, the spatial frequency information interaction network predicts and outputs the noise to be added of each time series according to the spatial features and frequency features; the noise to be added is sequentially added to the SAR image in the order of the time series to obtain a pure noise image; in the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed of each time series according to the pure noise image; the noise to be removed is removed from the pure noise image in the order of the time series to obtain a color image. The present invention successfully realizes the colorization processing of the SAR image by accurately extracting the spatial and frequency features of the SAR image, and using the spatial frequency information interaction network in the conditional diffusion model to perform forward denoising and reverse denoising processing, thereby improving the visual effect and practicality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] in: Figure 1 A schematic diagram of a colorization method for a SAR image in one embodiment; Figure 2 A schematic diagram of the structure of a spatial frequency information interaction network in one embodiment; Figure 3 Schematic diagram of the structure of a multi-scale spatial frequency interaction module MSFI in one embodiment; Figure 4 A schematic diagram of interactive calculation of spatial features and frequency features in one embodiment; Figure 5 The figure is a schematic diagram of the colorization process of a SAR image based on the interactive diffusion of spatial features and frequency features in one embodiment. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present invention; however, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details; in other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described, and it should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments presented here; on the contrary, providing these embodiments will make the disclosure thorough and complete and fully convey the scope of the present invention to those skilled in the art. The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "a", "an" and "said / the" are also intended to include the plural forms, unless the context clearly indicates otherwise, and it should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0021] Colorization technology can not only significantly improve the visual information content of SAR images, enhance the detailed features of images, and improve their intuitiveness, but also further improve the recognition accuracy of surface features on the basis of retaining the core space and radiation information of the image. This application proposes a colorization method for SAR images. Through colorization processing, the key information in the SAR image can be presented more intuitively, thereby helping professional analysts to more accurately analyze surface information.

[0022] In order to thoroughly understand the present invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by the present invention; optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementation methods.

[0023] like Figure 1 As shown, in one embodiment, a SAR image colorization method is provided, and the SAR image colorization method includes steps S101 to S106, which are described in detail as follows: S101, acquiring a SAR image, and extracting spatial features and frequency features of the SAR image; Extracting key spatial and frequency features from SAR images can provide the necessary information basis for subsequent processing.

[0024] In some embodiments, extracting the spatial features and frequency features of the SAR image specifically includes: Perform learnable wavelet transform on SAR images to obtain frequency features; The SAR image is downsampled to obtain spatial features.

[0025] In some embodiments, from the SAR image Extract frequency features from , which can be used as a conditional constraint to assist coloring in the image processing process, where the global frequency information extraction process can be expressed as follows:

[0026] In the formula, is a learnable wavelet transform layer, is the SAR image, is the frequency characteristic; The SAR image is downsampled according to the following formula to obtain the spatial features: :

[0027] In the formula, is the downsampling process, is the spatial feature, It is a SAR image.

[0028] Preferably, the above-mentioned spatial features can be , frequency characteristics Perform feature splicing to obtain fusion features for subsequent processing:

[0029] In the formula, is the feature fusion process, is the spatial feature, is the frequency characteristic, For fusion features.

[0030] S102, inputting the spatial features and the frequency features into a conditional diffusion model, wherein the conditional diffusion model includes a spatial frequency information interaction network, and performing forward denoising and reverse denoising on the SAR image through the spatial frequency information interaction network; Through the spatial frequency information interaction network, the SAR image features are effectively utilized, and the forward denoising and reverse denoising processes of the conditional diffusion model are initiated, providing an effective technical means for subsequent colorization processing.

[0031] like Figure 2 As shown, in some embodiments, the spatial frequency information interaction network includes a convolutional layer, two downsampling processing modules, a multi-scale spatial frequency interaction module MSFI, two upsampling processing modules and a convolutional layer connected in sequence; the upsampling processing module is composed of the multi-scale spatial frequency interaction module MSFI and upsampling, and the downsampling processing module is composed of the multi-scale spatial frequency interaction module MSFI and downsampling.

[0032] S103, in the forward noise adding process, the spatial frequency information interaction network predicts and outputs the noise to be added of each time series according to the spatial characteristics and frequency characteristics; The spatial-frequency information interaction network can accurately predict the noise that should be added to each time series based on the spatial and frequency characteristics of the input, and can provide precise guidance for generating pure noise images.

[0033] In some embodiments, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics, specifically including: The spatial features and frequency features are fused and input into the convolution layer to obtain the first output feature; Input the first output feature into the downsampling processing module of the first layer to obtain the second output feature; Input the second output feature into the downsampling processing module of the second layer to obtain a third output feature; The third output feature is input into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature; Input the fourth output feature into the upsampling processing module of the first layer to obtain the fifth output feature; Input the fifth output feature into the upsampling processing module of the second layer to obtain a sixth output feature; The sixth output feature is input into the convolution layer to obtain the noise to be added.

[0034] like Figure 3 As shown, in some embodiments, the multi-scale spatial frequency interaction module MSFI includes two learnable wavelet transform layers, two learnable inverse wavelet transform layers, three convolutional layers and two spatial frequency feature interaction layers SFIB.

[0035] In some embodiments, the third output feature is input into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature, specifically including: S301, inputting the third output feature into the learnable wavelet transform layer of the first layer and the convolution layer of the first layer respectively, obtaining corresponding first processing results and second processing results, wherein the first processing result includes a high-frequency component and a low-frequency component; S302, inputting the first processing result into the convolution layer of the second layer to obtain the first feature to be fused; S303, input the low-frequency component into the learnable wavelet transform layer of the second layer to obtain the secondary low-frequency component and input it into the convolution layer of the third layer to obtain the convolution result, and input the convolution result into the learnable wavelet transform layer of the first layer to obtain the second feature to be fused; S304, inputting the first feature to be fused and the second feature to be fused into the first spatial frequency feature interaction layer SFIB to obtain the interaction feature, and then inputting the interactive feature into the learnable wavelet transform layer of the second layer to obtain the third feature to be fused; S305 , input the third feature to be fused and the second processing result into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature.

[0036] Specifically, Figure 2 It includes five multi-scale spatial frequency interaction modules MSFI, and the processing flow of each multi-scale spatial frequency interaction module MSFI on the input features is consistent with steps S301-S305.

[0037] like Figure 3 As shown, in some embodiments, a given feature is input into a multi-scale spatial frequency interaction module MSFI to obtain a corresponding output feature, specifically including: Given input features , the first-level feature extraction result can be expressed as:

[0038]

[0039] In the formula, is the convolution process, is a learnable wavelet transform layer, is the feature result after the first-level feature extraction, For The variables after decomposition; For low frequency components Perform the second-level feature extraction, and the second-level feature extraction result can be expressed as:

[0040] In the formula, For Variables after decomposition.

[0041] The output of feature interaction can be expressed as:

[0042] In the formula, For the final output result, is the convolution operation, is a learnable wavelet transform layer, is a learnable inverse wavelet transform layer, It is the spatial-frequency feature interaction layer.

[0043] In some embodiments, the third feature to be fused and the second processing result are input into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature, specifically including: S301, inputting the third feature to be fused into the convolution layer to extract its corresponding spatial feature; S302, inputting the second processing result into a learnable inverse wavelet transform layer to obtain its corresponding frequency feature; S303, performing feature expansion on the spatial features and the frequency features to obtain corresponding first sequence features and second sequence features; S304, performing feature mapping and position encoding on the first sequence features and the second sequence features based on a multi-layer perceptron MPL to obtain corresponding first encoding features and second encoding features; S305, mapping the first coding feature and the second coding feature into a plurality of feature vectors respectively and obtaining vector features calculated by interaction of the first coding feature and the second coding feature with the frequency sequence feature; S306, mapping the vector features into a feature map; S307. Determine a fourth output feature of the interactive calculation according to the feature graph.

[0044] Specifically, Figure 3 It includes two spatial frequency feature interaction layers SFIB, and the processing flow of each spatial frequency feature interaction layer SFIB on the input feature is consistent with steps S301-S307.

[0045] like Figure 4 As shown, in some embodiments, the spatial domain features and frequency features are input into the spatial frequency feature interaction layer SFIB to obtain corresponding output features, specifically including: Spatial features extracted by convolutional layers And the frequency feature extraction results obtained by the learnable inverse wavelet transform layer As the input features of the spatial frequency feature interaction layer SFIB, the above features are expanded (Img2Seq) to obtain the sequence features Used for subsequent interactive calculations; Based on multi-layer perceptron MPL and Perform feature mapping and position encoding (Position embedding PE) of the sequence feature information to obtain the encoded features and ; Position-encoded sequence features As an example, map it into three eigenvectors . Then with the frequency sequence characteristics The interactive calculation process can be expressed as:

[0046] The vector features after interactive calculation can be obtained by learning weights Calculated; Calculate the frequency sequence characteristics Interaction Results Finally, the sequence features after the space-frequency domain interaction are mapped from sequence to feature graph (Seq2Img) to obtain the feature graph after the interaction, and then element-by-element addition is performed to obtain the final interactive calculation output features.

[0047] S104, adding the noise to be added to the SAR image in sequence according to the order of the time series to obtain a pure noise image; By gradually adding the predicted noise, a pure noise image corresponding to the SAR image was successfully generated, which provided a basis for subsequent denoising processing.

[0048] Specifically, the original data is converted into pure noise through a series of steps. In each step, a small amount of controllable Gaussian noise is added to the image, making it gradually distorted and close to random noise. The forward noise diffusion process can be expressed as follows:

[0049] In the formula, is the current time step The image status, For the previous time step The image state is The first step of adding noise provides a starting point, is a variable that controls the noise variance, is the identity matrix, when it is When multiplied, we can determine that the uniform addition to The variance of the noise in each element.

[0050] S105, in the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed in each time series according to the pure noise image; The spatial-frequency information interaction network also performs well in the inverse denoising stage and can accurately predict the noise that should be removed from each time series, providing strong support for restoring color images.

[0051] Specifically, the noise level is iteratively reduced, the original data is reconstructed from the noise, and the SAR image is colored. The inverse denoising process can be expressed as follows:

[0052] In the formula, is a function used to predict The mean of the distribution, is a function that determines The variance of the distribution.

[0053] S106 , removing the noise to be removed from the pure noise image in the order of time series to obtain a color image.

[0054] By gradually removing the predicted noise, a color image can be successfully restored, and the color image has richer color information and higher clarity than the original SAR image in terms of visual effect.

[0055] In summary, if Figure 5 As shown in the figure, the dotted box represents the conditional diffusion model, which includes the forward denoising process and the reverse denoising process. The solid box represents the space-frequency information interaction network, which is closely connected with the conditional diffusion model. The figure describes the entire processing flow from the input SAR image to the final colorized SAR image output: (1) Obtaining prior conditions: From SAR images In this paper, the frequency features are extracted by learnable wavelet transform. , downsample the original input SAR image to obtain the spatial features ; (2) Conditional Diffusion Model: For the above spatial features , frequency characteristics Perform feature splicing to obtain fusion features .

[0056] The fusion features Input the conditional diffusion model and perform the forward noise diffusion process and the reverse denoising process: Forward noise diffusion process: The original SAR image is converted into a pure noise image by gradually adding controllable Gaussian noise.

[0057] Reverse denoising process: The noise is gradually removed through an iterative optimization algorithm, the original data is reconstructed from the pure noise image, and finally a colorized SAR image is obtained.

[0058] (3) Spatial frequency information interaction network: The space-frequency information interaction network runs through the entire conditional diffusion model, realizing the interaction between space and frequency information, making full use of the characteristics of local spatial information and global frequency information, thereby significantly improving the colorization performance of SAR images.

[0059] Finally, after being processed by the conditional diffusion model, the original SAR images were successfully converted into colorized images. These colorized images have higher visual clarity and information content, which helps professional analysts to interpret surface information more accurately. Figure 1 On the right side, the colorized SAR image output is shown.

[0060] The present application also proposes a SAR image colorization method system, the system comprising: an image acquisition unit, a feature extraction unit and an image processing unit; An image acquisition unit, used for acquiring a SAR image; A feature extraction unit, used to extract spatial features and frequency features of an input SAR image; The image processing unit includes a space-frequency information interaction network, a denoising module and a denoising module, and is used to perform forward denoising and reverse denoising on the SAR image, wherein: Spatial-frequency information interaction network: predicts and outputs the noise to be added and removed for each time series based on spatial and frequency characteristics; Noise adding module: according to the noise to be added output by the space-frequency information interaction network, it is added to the SAR image in the order of time series to obtain a pure noise image; Denoising module: According to the noise to be removed output by the space-frequency information interaction network, the noise is removed from the pure noise image in the order of the time series to obtain a color image.

[0061] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps: Acquire SAR images and extract spatial and frequency features of SAR images; The spatial features and frequency features are input into a conditional diffusion model, the conditional diffusion model includes a spatial frequency information interaction network, and the SAR image is forward denoised and reverse denoised through the spatial frequency information interaction network; In the forward noise addition process, the spatial-frequency information interaction network predicts and outputs the noise to be added to each time series based on the spatial and frequency characteristics; The noise to be added is sequentially added to the SAR image in the order of time series to obtain a pure noise image; In the inverse denoising process, the spatial-frequency information interaction network predicts and outputs the noise to be removed in each time series based on the pure noise image; The noise to be removed is removed from the pure noise image in the order of time series to obtain a color image.

[0062] The present application also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to perform the following steps: Acquire SAR images and extract spatial and frequency features of SAR images; The spatial features and frequency features are input into a conditional diffusion model, the conditional diffusion model includes a spatial frequency information interaction network, and the SAR image is forward denoised and reverse denoised through the spatial frequency information interaction network; In the forward noise addition process, the spatial-frequency information interaction network predicts and outputs the noise to be added to each time series based on the spatial and frequency characteristics; The noise to be added is sequentially added to the SAR image in the order of time series to obtain a pure noise image; In the inverse denoising process, the spatial-frequency information interaction network predicts and outputs the noise to be removed in each time series based on the pure noise image; The noise to be removed is removed from the pure noise image in the order of time series to obtain a color image.

[0063] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. The above disclosure is only a preferred embodiment of the present invention, and of course it cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present invention.

Claims

1. A method for colorizing a SAR image, characterized in that: The method comprises: Acquire a SAR image, and extract spatial features and frequency features of the SAR image; Inputting the spatial features and frequency features into a conditional diffusion model, wherein the conditional diffusion model includes a spatial frequency information interaction network, and performing forward denoising and reverse denoising on the SAR image through the spatial frequency information interaction network; In the forward noise addition process, the spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics; Adding the noise to be added to the SAR image in sequence according to the order of the time series to obtain a pure noise image; In the reverse denoising process, the spatial frequency information interaction network predicts and outputs the noise to be removed of each of the time series according to the pure noise image; The noise to be removed is removed from the pure noise image in the order of the time series to obtain a color image.

2. The SAR image colorization method according to claim 1, characterized in that: The extracting of the spatial features and frequency features of the SAR image specifically includes: Performing a learnable wavelet transform on the SAR image to obtain frequency features; The SAR image is downsampled to obtain spatial features.

3. The SAR image colorization method according to claim 1, characterized in that: The spatial frequency information interaction network includes a convolution layer, two downsampling processing modules, a multi-scale spatial frequency interaction module MSFI, two upsampling processing modules and a convolution layer connected in sequence; the upsampling processing module is composed of a multi-scale spatial frequency interaction module MSFI and upsampling, and the downsampling processing module is composed of a multi-scale spatial frequency interaction module MSFI and downsampling.

4. The method for colorizing a SAR image according to claim 3, characterized in that: The spatial frequency information interaction network predicts and outputs the noise to be added to each time series according to the spatial characteristics and frequency characteristics, specifically including: The spatial feature and the frequency feature are fused and input into a convolution layer to obtain a first output feature; Inputting the first output feature into a downsampling processing module of the first layer to obtain a second output feature; Inputting the second output feature into the downsampling processing module of the second layer to obtain a third output feature; Input the third output feature into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature; Inputting the fourth output feature into the upsampling processing module of the first layer to obtain a fifth output feature; Inputting the fifth output feature into the upsampling processing module of the second layer to obtain a sixth output feature; The sixth output feature is input into the convolution layer to obtain the noise to be added.

5. The method for colorizing a SAR image according to claim 4, characterized in that: The multi-scale spatial frequency interaction module MSFI includes two learnable wavelet transform layers, two learnable inverse wavelet transform layers, three convolutional layers and two spatial frequency feature interaction layers SFIB.

6. The method for colorizing a SAR image according to claim 4, characterized in that: Inputting the third output feature into a multi-scale spatial frequency interaction module MSFI to obtain a fourth output feature specifically includes: Inputting the third output feature into the learnable wavelet transform layer of the first layer and the convolution layer of the first layer respectively, obtaining corresponding first processing results and second processing results, wherein the first processing result includes a high-frequency component and a low-frequency component; Inputting the first processing result into the convolution layer of the second layer to obtain the first feature to be fused; Input the low-frequency component into the learnable wavelet transform layer of the second layer to obtain a secondary low-frequency component and input it into the convolution layer of the third layer to obtain a convolution result, and input the convolution result into the learnable wavelet transform layer of the first layer to obtain a second feature to be fused; The first feature to be fused and the second feature to be fused are input into the first spatial frequency feature interaction layer SFIB to obtain the interaction feature, and then the interaction feature is input into the learnable wavelet transform layer of the second layer to obtain the third feature to be fused; The third feature to be fused and the second processing result are input into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature.

7. The method for colorizing a SAR image according to claim 6, characterized in that: The step of inputting the third feature to be fused and the second processing result into the second spatial frequency feature interaction layer SFIB to obtain the fourth output feature specifically includes: Input the third feature to be fused into the convolution layer to extract its corresponding spatial feature; Inputting the second processing result into a learnable inverse wavelet transform layer to obtain the corresponding frequency feature; Perform feature expansion on the spatial features and the frequency features to obtain corresponding first sequence features and second sequence features; Perform feature mapping and position encoding on the first sequence features and the second sequence features based on a multi-layer perceptron MPL to obtain corresponding first encoding features and second encoding features; Mapping the first coding feature and the second coding feature into a plurality of feature vectors respectively and obtaining vector features calculated by interaction of the first coding feature and the second coding feature with the frequency sequence feature; Mapping the vector features into a feature map; A fourth output feature of the interactive calculation is determined according to the feature graph.

8. A SAR image colorization method system, characterized in that: The system comprises: an image acquisition unit, a feature extraction unit and an image processing unit; The image acquisition unit is used to acquire the SAR image; The feature extraction unit is used to extract the spatial features and frequency features of the input SAR image; The image processing unit includes a space-frequency information interaction network, a denoising module and a denoising module, which are used to perform forward denoising and reverse denoising on the SAR image, wherein: Spatial-frequency information interaction network: predicts and outputs the noise to be added and removed for each time series based on spatial and frequency characteristics; Noise adding module: according to the noise to be added output by the space-frequency information interaction network, it is added to the SAR image in the order of time series to obtain a pure noise image; Denoising module: According to the noise to be removed output by the space-frequency information interaction network, the noise is removed from the pure noise image in the order of the time series to obtain a color image.

9. A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

Images