A synthetic aperture radar (SAR) image colorization system

By employing a diffusion-based synthetic aperture radar image colorization system, and utilizing a cross-attention mechanism and a three-step gradient method, the problems of content mapping errors and computational complexity in SAR image colorization are solved. This results in the generation of accurate, detailed, and structurally clear optical images, improving the semantic consistency and inference efficiency of the generated images.

CN120472026BActive Publication Date: 2026-01-13GUANGDONG UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510543529.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-04-28
Publication Date
2026-01-13
Estimated Expiration
2045-04-28

AI Technical Summary

Technical Problem

Existing SAR image colorization methods suffer from problems such as content mapping errors, color shifts, and missing spatial information. Furthermore, traditional diffusion models have high computational complexity, resulting in long inference times and limiting their application in SAR image colorization.

Method used

A synthetic aperture radar image colorization system based on a diffusion model is adopted. A multimodal dataset is acquired through a data acquisition module, and a noise prediction model is trained using a cross-attention mechanism network. Semantic conditions are incorporated into the training process, and a three-step gradient method is used to accelerate image generation and ensure the consistency between the generated image and the target semantics.

Benefits of technology

More accurate, detailed and structurally clear optical images were generated, significantly improving the consistency between the generated images and the target semantics, reducing computational complexity, improving inference efficiency, and enhancing the semantic accuracy and practicality of the generated images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120472026B_ABST
    Figure CN120472026B_ABST
Patent Text Reader

Abstract

The application discloses a synthetic aperture radar (SAR) image colorization system, comprising a data acquisition module, a model training module and a colorized image generation module; the data acquisition module is used for acquiring a multi-modal data set; the model training module is used for training a noise prediction model by using the multi-modal data set, wherein the noise prediction model is obtained by constructing a cross-attention mechanism network; and the colorized image generation module is used for inputting a to-be-predicted image into the noise prediction model to obtain a color optical image. The application is suitable for disaster monitoring, landform surveying and other scenes requiring high-precision SAR image analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image technology, and particularly relates to a synthetic aperture radar (SAR) image colorization system. Background Technology

[0002] Given the visual unfriendliness of SAR imagery compared to other optical imagery, SAR image colorization has become a crucial research area for enhancing SAR image interpretation. Traditional SAR colorization methods rely on machine learning and image processing techniques, requiring the classification of ground targets and the construction of a classification translation knowledge base, as well as the application of feature extraction algorithms to achieve feature transformation and obtain mapping relationships between various targets. These methods are costly to implement, knowledge-dependent, and difficult to generalize to complex and varied ground scenes and detailed visualization interpretation. With the development of deep learning, especially the emergence of Generative Adversarial Networks (GANs), the development of grayscale image colorization methods has been greatly promoted. Compared to traditional feature mapping-based methods, GANs can reduce human intervention and achieve automatic colorization through end-to-end learning. However, the fundamental differences in imaging mechanisms between SAR and optical images mean that GAN-based colorization methods still suffer from problems such as content mapping errors, color shifts, and missing spatial information. Furthermore, GAN training is prone to mode collapse and training instability, making model optimization difficult and thus limiting its further application in SAR image colorization research.

[0003] Against this backdrop, diffusion models have gradually become a more ideal solution for SAR image colorization due to their stronger image generation capabilities, more stable training process, and better detail restoration capabilities. Studies have shown that in many computer vision tasks, diffusion models outperform GANs in image quality and can more stably learn the mapping relationship from SAR to optical images. However, directly applying diffusion models to SAR colorization still faces challenges. Because the backscattering characteristics of SAR images differ from the spectral reflectance characteristics of optical images, existing diffusion models, when directly applied to SAR image colorization, exhibit deviations in content mapping and color mapping, resulting in significant differences in color distribution and content consistency between the colorized image and the real optical image. Furthermore, traditional diffusion models use a stepwise denoising approach for image generation, which has high computational complexity and leads to long inference times, limiting their practicality. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a synthetic aperture radar (SAR) image colorization system, suitable for scenarios requiring high-precision SAR image analysis, such as disaster monitoring and geomorphological surveys.

[0005] This invention provides a synthetic aperture radar (SAR) image colorization system, comprising: a data acquisition module, a model training module, and a colorized image generation module;

[0006] The data acquisition module is used to acquire multimodal datasets;

[0007] The model training module is used to train a noise prediction model using the multimodal dataset, wherein the noise prediction module is constructed through a cross-attention mechanism network;

[0008] The colorization image generation module is used to input the image to be predicted into the noise prediction model to obtain a color optical image.

[0009] Optionally, obtaining a multimodal dataset includes:

[0010] Acquire SAR and optical images;

[0011] The optical image is processed to obtain text vectors and a noisy image;

[0012] The multimodal dataset is obtained based on the SAR image, text vector, and noisy image.

[0013] Optionally, processing the optical image to obtain a text vector includes:

[0014] A preliminary textual description of the optical image is extracted using the BLIP model;

[0015] The preliminary text description is fused with parametric metadata from remote sensing imagery to obtain a complete text description;

[0016] The complete text description is encoded to obtain the text vector.

[0017] Optionally, processing the optical image to obtain a noisy image includes:

[0018] The optical image is subjected to noise reduction processing to obtain the noise-reduced image.

[0019] Optionally, the cross-attention mechanism network is used to acquire fused features.

[0020] Optionally, the method for obtaining the fusion features includes:

[0021] The SAR image is used as conditional information and concatenated with the noisy image in the channel dimension to obtain joint features. The joint features are then used as the query vector Q.

[0022] Using the text vector as the initial key K and initial value V, the fused features are obtained by matching the query vector Q and the initial key K, and combining them with the initial value V.

[0023] Optionally, constructing the noise prediction model further includes: training the noise prediction model using a multi-objective loss function, wherein the multi-objective loss function is:

[0024]

[0025] in, For multi-objective loss functions, For image reconstruction loss, In order to perceive loss, λ1, λ2, and λ3 are the semantic consistency loss and the loss weights.

[0026] Optionally, inputting the image to be predicted into the noise prediction model to obtain a color optical image includes:

[0027] Initialize a Gaussian noise image that matches the preset size of the SAR image, and set the initial time step;

[0028] Based on the initial time step, the Gaussian noise image is denoised using a three-step gradient to obtain the color optical image.

[0029] Optionally, denoising the Gaussian noise image using a three-step gradient descent method to obtain the color optical image includes:

[0030] Based on the initial time step, calculate the first gradient at time step t, and advance by half a time step to time t. Obtain the second gradient and advance three-quarters of the time steps to Obtain the third gradient;

[0031] Based on the first gradient, the second gradient and the third gradient, update the denoised Gaussian noise image and obtain the Gaussian noise image at time t.

[0032] Starting from the initial time step, the time step t is gradually reduced until the time step t is 0, at which point the color optical image is acquired.

[0033] Compared with the prior art, the present invention has the following advantages and technical effects:

[0034] This invention enables the generation of more accurate, detailed, and structurally clear optical images based on SAR images. By introducing semantic conditions into the training process, the consistency between the generated image and the target semantics is significantly improved, effectively avoiding the semantic bias problem that occurs in traditional methods under limited image quality conditions. For example, desert areas in some low-quality SAR images are easily misidentified as oceans, while this invention, through a semantic prior guidance mechanism, effectively suppresses such misidentification and significantly enhances the semantic accuracy of the generated images.

[0035] Furthermore, this invention, based on a diffusion model—a high-performance image generation framework—possesses strong detail restoration capabilities, generating more realistic and natural optical images in terms of texture, edges, and color transitions. Moreover, a three-step gradient acceleration method is employed for feature estimation, effectively reducing computational complexity. While maintaining only a slight decrease in generation quality, inference efficiency is significantly improved, making this method highly practical and valuable for widespread application. Attached Figure Description

[0036] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a flowchart of a synthetic aperture radar (SAR) image colorization system according to an embodiment of the present invention;

[0038] Figure 2 This is a flowchart illustrating the training process of the cross-attention mechanism network noise prediction model according to an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] This embodiment proposes a synthetic aperture radar (SAR) image colorization system, including: a data acquisition module, a model training module, and a colorized image generation module;

[0042] The data acquisition module is used to acquire multimodal datasets;

[0043] The model training module is used to train a noise prediction model using a multimodal dataset. The noise prediction module is constructed through a cross-attention mechanism network.

[0044] The colorization image generation module is used to input the image to be predicted into the noise prediction model to obtain a color optical image.

[0045] Specifically, the system is designed based on the EDM (Elucidating Diffusion Models) diffusion model and mainly consists of two parts: training and generation. In the training phase, a BLIP model pre-trained based on remote sensing data is first used to extract scene text descriptions from the optical images in the dataset. Then, parameters such as satellite model and polarization type are incorporated into the text descriptions. The generated text data is used to constrain the consistency of scene land cover types after SAR image colorization, ensuring that the colorization result matches the real land cover categories. Next, the pre-trained CLIP model is used to encode the text descriptions to obtain corresponding text vectors. The noisy optical image, time step (noise intensity), and scene information text vectors are then input into a noise prediction network for training. To further improve the quality of colorization and land cover consistency, the noise prediction network introduces a text-guided cross-attention mechanism to enhance the constraint of text information on the generated results. Furthermore, the system constructs a multi-objective loss function, including image reconstruction loss, color consistency loss, and semantic consistency loss, and optimizes the model parameters through backpropagation to finally obtain the colorization model. In the system model generation phase, a Gaussian noise image matching the SAR image size is first initialized, and an initial time step T is set. Subsequently, a well-trained SAR colorization model is used, with SAR images serving as the colorization guide. During denoising, the Bogacki-Shampine method is employed, dynamically adjusting the denoising direction at different time steps t by calculating three intermediate gradients, and updating the image using a weighted averaging strategy. This reduces the number of generation steps and accelerates the denoising process through more accurate denoising estimation. Finally, after multiple iterations, a colorized result conforming to the SAR image structure and text description is generated.

[0046] Furthermore, obtaining the multimodal dataset includes:

[0047] Acquire SAR and optical images;

[0048] The optical image is processed to obtain text vectors and noisy images;

[0049] A multimodal dataset is obtained based on SAR images, text vectors, and noisy images.

[0050] Specifically, the BLIP-RemoteGLM model, trained on remote sensing data, generates textual descriptions of optical image scene types. This is combined with imaging parameters from the remote sensing imagery (such as optical satellite type, SAR polarization, and SAR imaging mode) to ensure consistency in the colorization results across ground cover categories, while also enabling the system to generate images for specific satellite types. In detail, the BLIP-RemoteGLM model first extracts preliminary textual descriptions from the optical images, then integrates parametric metadata from the remote sensing imagery to form complete textual descriptions. Ultimately, the dataset is expanded from SAR images and optical images to include SAR images, optical images, and their corresponding textual descriptions.

[0051] Further processing of the optical image to obtain text vectors includes:

[0052] Preliminary textual descriptions of optical images are extracted using the BLIP model;

[0053] The preliminary text description is fused with parametric metadata from remote sensing imagery to obtain a complete text description;

[0054] Encode the complete text description to obtain a text vector.

[0055] Further processing of the optical image to obtain a noisy image includes:

[0056] The optical image is noise-added to obtain a noisy image.

[0057] Furthermore, the noise prediction model includes: a residual module, an upsampling module, a downsampling module, and an attention module;

[0058] The residual module enhances the ability to model nonlinear features, thereby improving training stability and model depth.

[0059] The upsampling module improves resolution through deconvolution, interpolation, and other methods for image restoration.

[0060] The downsampling module reduces the resolution through convolution or pooling to extract higher-level semantic information.

[0061] The attention module models global dependencies within an image, enhancing the representation of image features.

[0062] The noise prediction model is a Unet-Shaped network. Similar to Unet, its main structure consists of a downsampling branch, a bottleneck, and an upsampling branch. The downsampling branch has 5 layers, and the upsampling branch also has 5 layers.

[0063] The first two layers of the downsampling branch consist of a residual module and a downsampling module, while the last three layers consist of a residual module, a self-attention module, and a downsampling module.

[0064] The bottleneck layer in the middle consists of a residual module, a cross-attention module, and a residual module. The cross-attention module is used to further interact and fuse features from text and images, guiding the semantic consistency of the generated image.

[0065] The first three layers of the upsampling branch consist of a residual module, a self-attention module, and an upsampling module. The last two layers consist of a residual module and an upsampling module.

[0066] In addition, the input of each layer of the upsampling branch is obtained by concatenating the output of the previous layer and the output of the corresponding layer of the downsampling branch, so that the final output can better integrate the information of the input image.

[0067] Specifically, the "noise prediction network" mentioned in this embodiment is a deep network with a structure similar to U-Net, including an Encoder part and a Decoder part. It further includes residual blocks, upsampling layers, downsampling layers, and attention blocks. The attention blocks include self-attention network layers and cross-attention network layers. Skip connections can fuse the shallow detail features of the encoder with the deep semantic features of the decoder through concatenation, thereby enabling the network to better utilize contextual information.

[0068] The "text-guided cross-attention mechanism" mentioned in this embodiment is a submodule of the BottleNeck module integrated into the noise prediction network.

[0069] Furthermore, a cross-attention mechanism network is used to acquire fused features.

[0070] Furthermore, the features used to obtain the fusion features include:

[0071] The SAR image is used as conditional information and concatenated with the noisy image in the channel dimension to obtain joint features. The joint features are then used as the query vector Q.

[0072] Using the text vector as the initial key K and initial value V, the fused features are obtained by matching the query vector Q and the initial key K, and combining them with the initial value V.

[0073] Specifically, the design and use of a text-guided cross-attention mechanism is characterized by the interaction of Query(Q), Key(K), and Value(V) to make the matching of SAR images and colorized optical images in different regions more accurate, improving the consistency of the scene before and after colorization. Query(Q) is influenced by the joint features of the current optical image and SAR image. Key(K) combines text features to define the target that should be matched in different regions. Value(V) mainly consists of text features, providing scene information from the optical image to guide the colorization result to conform to the semantic features of the target scene. Specifically, firstly, the joint features in the optical image and SAR image are obtained as Q from the encoder output of the noise prediction network. Next, the text encoding result from step two is used as the initial K and V. Finally, by matching the regions in Q and K and combining the scene information provided by V, cross-attention is calculated to adjust the image generation process, so that the final generated optical image conforms to the consistency requirements of the target scene.

[0074] Furthermore, constructing the noise prediction model also includes training the noise prediction model using a multi-objective loss function, wherein the multi-objective loss function is:

[0075]

[0076] in, For multi-objective loss functions, For image reconstruction loss, In order to perceive loss, λ1, λ2, and λ3 are the semantic consistency loss and the loss weights.

[0077] Specifically, the characteristic of constructing the multi-objective loss function is that the constructed loss function includes image reconstruction loss, color consistency loss, and semantic consistency loss. Image reconstruction uses L2 loss to measure the pixel-level difference between the generated image and the target optical image, ensuring consistency of structural details and optimizing the overall structure and texture. Color consistency loss extracts features through a pre-trained VGG network to measure the feature differences between the generated image and the target optical image at different levels, constraining color consistency from the feature space and reducing color cast. Semantic consistency loss utilizes CLIP or other semantic matching models to calculate the semantic similarity between the generated image and the target text description, improving the alignment between the image and text.

[0078] Further, the image to be predicted is input into the noise prediction model to obtain a color optical image, including:

[0079] Initialize a Gaussian noise image that matches the preset size of the SAR image, and set the initial time step;

[0080] Based on the initial time step, a three-step gradient is used to denoise the Gaussian noise image and obtain a color optical image.

[0081] Specifically, the denoising calculation step in the generation process is characterized by: employing the Bogacki-Shampine method to calculate three intermediate gradients, and accelerating the denoising process through weighted averaging. Specifically, by probing future states in stages (half-step and three-quarter-step), local biases from single-step predictions are avoided, and the traditional solver can be directly replaced without additional training. This simplifies the generation process and improves computational efficiency.

[0082] Furthermore, the Gaussian noise image is denoised using a three-step gradient method to obtain a color optical image, including:

[0083] Based on the initial time step, calculate the first gradient at time step t, and advance by half a time step to time t. Obtain the second gradient and advance three-quarters of the time steps to Obtain the third gradient;

[0084] Based on the first gradient, the second gradient, and the third gradient, update the denoised Gaussian noise image and obtain the Gaussian noise image at time t.

[0085] Starting from the initial time step, the time step t is gradually reduced until the time step t is 0, at which point a color optical image is acquired.

[0086] Specifically, in the inference phase of the designed diffusion model, the first step is to start with Gaussian noise. Starting with the input time step T, the denoised SAR image, and parameter text information as guidance, a fully trained noise prediction model is used to predict Gaussian noise x. T Denoising and generation are performed, and the Bogacki-Shampine method is introduced to accelerate the inverse generation process of the diffusion model and reduce the number of iterations. Finally, the strategy used in the training phase is used for prediction, and this process is repeated until t = 0, at which point a colorized image is output.

[0087] The following is a detailed description of this embodiment with reference to the accompanying drawings:

[0088] This embodiment provides a synthetic aperture radar (SAR) image colorization system, such as... Figure 1-2 As shown, the specific steps are as follows:

[0089] Step 1: In the training phase of the designed diffusion model, the RemoteGLM model trained based on remote sensing data is first used to generate text descriptions of optical images. Parameters such as satellite model, polarization mode, and imaging mode are integrated into the text descriptions. The texts are connected with commas. The generated text descriptions will serve as a consistency constraint for the scene and ground cover types after the SAR image is colored, to prevent the problem of failing to generate a scene that matches the SAR image after the SAR image is colored.

[0090] Specifically, reading optical images I optical The BLIP-RemoteGLM model, fine-tuned based on remote sensing domain knowledge, is used to perform scene understanding on optical images and generate corresponding preliminary text descriptions. Caption , i.e., text Caption =BLIP(I optical Next, the satellite model, polarization type, imaging model, and other parameter information are compiled into a text file. param This is integrated into the initial text description to improve adaptability to SAR imaging characteristics. Ultimately, based on text=text Caption +text param , obtain complete scene description text and complete parameter information text.

[0091] Step 2: Next, the CLIP model with frozen parameters is used to encode the text description into a 512-dimensional vector. A small neural network is then built to map and transform the 512-dimensional text vector output by the CLIP model. By performing a linear transformation on these text features, the network adjusts them to a dimension that matches the feature space of the diffusion model, thereby ensuring that the text vector can be effectively fused with image features (such as SAR images or generated optical images).

[0092] Specifically, the CLIP model using frozen parameters is determined by c Text =CLIP text (Text) generates a 512-dimensional text vector c. Text As a subsequent conditional guide, a neural network with an MLP-Relu-MLP structure is then built to enhance features and perform dimensionality transformation.

[0093] Step 3: Add noise using a time-proportional noise reduction strategy. The traditional EDM model's noise reduction strategy is x. t = s(t)x0 + σ(t)ε, where s(t) and σ(t) are functions of time step t, controlling the influence of the original image x0 and Gaussian noise ε, respectively. Here, we assume s(t) = 1, σ(t) = t, and the noise addition formula is x t= x0 + tε, where the noise intensity increases linearly with the time step, and the noise intensity can be directly reflected by the time step. Input samples N with different noise levels are generated based on this formula. optical To proceed to the next step of noise prediction model U θ Training.

[0094] Step 4: The noisy optical image, the time step (noise intensity), and the scene information text vector are jointly input into the noise prediction network for training. The SAR image, as conditional information, is concatenated with the noisy optical image along the channel dimension to form joint features. This allows the network to learn the noise information of the optical image and the geometric and textural features of the SAR image, which are then used to generate the query vector (Q) in the cross-attention mechanism. On the other hand, the scene text description, after encoding, is mapped to a feature space matching the time step (noise intensity) and used to regulate the generation dynamics during the diffusion process. The text description, after encoding and dimensional transformation, is embedded into the time step (noise intensity) to regulate the generation dynamics during the diffusion process. Simultaneously, the text features, after transformation, generate the key K and value V required for the cross-attention mechanism, enhancing the text guidance capability. In the cross-attention mechanism, by matching regions in Q (SAR-optical joint features) and K (text features), and based on the scene information provided by V, the denoising network is guided to generate an optical image that conforms to the SAR structure and text semantics. This ensures that the generated optical image visually matches the SAR image and conforms to the given scene description.

[0095] Specifically, to improve the consistency of scene information during SAR image colorization, a text-guided cross-attention mechanism is introduced into the noise prediction network, enabling the model to utilize textual information to guide the generation of the colorized image. The noise prediction network U... θ The input includes: the noisy optical image N optical SAR image I sar Text vector c Text Time step (noise intensity) t. Based on Q = W Q ·(f([I sar N optical The query Q can be calculated by stitching together SAR and noisy optical images, where f represents the feature extraction part in the denoising network. The key (K) and value (V) are calculated from the scene text: K = W K ·c text V = W V ·c text W Q W K W VThis is a learnable projection matrix used to map input features to the same feature space for calculating attention weights. The above variables are processed through a text-guided cross-attention mechanism and calculated as follows. The final fused features are obtained, containing SAR-optical information and semantic scene information. Finally, the network is based on I... output =U θ (N optical ,I SAR ,c text The final output is the predicted image.

[0096] Step 5: Design a multi-objective loss function, constructing a joint loss function consisting of image reconstruction loss, color consistency loss, and semantic consistency loss. The image reconstruction loss is... Color consistency loss is composed of perceptual loss. Features are extracted using a pre-trained VGG network to measure the feature differences between the generated image and the target optical image at different levels, constraining color consistency from the feature space and reducing color cast. Defined as... Semantic consistency utilizes CLIP or other semantic matching models to calculate the semantic similarity between the generated image and the target text description, improving the alignment between the image and text. It is defined as follows: Here, ψ(·) is a pre-trained CLIP2 model that maps both images and text to the same feature space. A multi-objective loss function is constructed based on this. Where λ1 = 1.0, λ2 = 0.5, and λ3 = 0.1. Finally, the model parameters are updated through backpropagation and an optimizer to make the generated image more consistent with the target optical image and text description in terms of detail, color, and semantics.

[0097] Step Six: As Figure 1 As shown, the denoised SAR image I SAR With text vector param Input the trained noise prediction network In the middle, based on the generation formula The Bogacki-Shampine method is used to accelerate the inversion process. Specifically, Gaussian noise is first initialized at time step T. Then from x T Begin denoising gradually, with each step's state denoted as x. t The denoising process is accelerated through a three-step gradient calculation. First, the gradient at time t is calculated. Then advance by half a step to Predict intermediate states and re-evaluate gradients to obtain new gradients. Next, advance to three-quarters of the time. Further refine the gradient direction to obtain the gradient. Finally, combining the gradient results from the three iterations, based on... Update the denoised state to obtain the image at the next time step. Repeat this process, starting from T and gradually decreasing t by Δt each time, until t = 0, finally obtaining the denoised optical image x0, i.e., the SAR image I. SAR With text vector param Guided color optical image I color .

[0098] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A synthetic aperture radar (SAR) image colorization system, characterized in that, include: Data acquisition module, model training module, and color image generation module; The data acquisition module is used to acquire multimodal datasets; Obtaining a multimodal dataset includes: Acquire SAR and optical images; The optical image is processed to obtain text vectors and a noisy image; The multimodal dataset is obtained based on the SAR image, text vector, and noisy image. The model training module is used to train a noise prediction model using the multimodal dataset, wherein the noise prediction module is constructed through a cross-attention mechanism network; Constructing the noise prediction model further includes: training the noise prediction model using a multi-objective loss function, wherein the multi-objective loss function is: in, For multi-objective loss functions, For image reconstruction loss, In order to perceive loss, For semantic consistency loss, , , For loss weights; The colorization image generation module is used to input the image to be predicted into the noise prediction model to obtain a color optical image; Inputting the image to be predicted into the noise prediction model to obtain a color optical image includes: Initialize a Gaussian noise image with the same preset size as the SAR image, and set the initial time step; Based on the initial time step, the Gaussian noise image is denoised using a three-step gradient to obtain the color optical image; Denoising the Gaussian noise image using a three-step gradient descent method to obtain the color optical image includes: Based on the initial time step, calculate the first gradient at time step t, and advance by half a time step to time t. Obtain the second gradient and advance three-quarters of the time steps to Obtain the third gradient; Based on the first gradient, the second gradient and the third gradient, update the denoised Gaussian noise image and obtain the Gaussian noise image at time t. Starting from the initial time step, the time step t is gradually reduced until the time step t is 0, at which point the color optical image is acquired.

2. The synthetic aperture radar (SAR) image colorization system according to claim 1, characterized in that, Processing the optical image to obtain text vectors includes: A preliminary textual description of the optical image is extracted using the BLIP model; The preliminary text description is fused with parametric metadata from remote sensing imagery to obtain a complete text description; The complete text description is encoded to obtain the text vector.

3. A synthetic aperture radar (SAR) image colorization system according to claim 1, characterized in that, Processing the optical image to obtain a noisy image includes: The optical image is subjected to noise reduction processing to obtain the noise-reduced image.

4. A synthetic aperture radar (SAR) image colorization system according to claim 1, characterized in that, The cross-attention mechanism network is used to obtain fused features.

5. A synthetic aperture radar (SAR) image colorization system according to claim 4, characterized in that, Obtaining the fusion features includes: The SAR image is used as conditional information and concatenated with the noisy image in the channel dimension to obtain joint features. The joint features are then used as the query vector Q. Using the text vector as the initial key K and initial value V, the fused features are obtained by matching the query vector Q and the initial key K, and combining them with the initial value V.

Citation Information

Patent Citations

  • SAR-optical image fusion method based on hybrid model

    CN117314811A

  • Typical ground object target segmentation method based on multi-source data attention feature fusion

    CN119131374A