A method for constructing a synthetic aperture radar image conversion network

Abstract

This invention discloses a method for constructing a synthetic aperture radar (SAR) image conversion network, belonging to the field of SAR image processing technology. It is used to convert SAR images into optical images. The method includes constructing a SAR image conversion network based on a generative adversarial network (GAN) framework. The network includes a generator and a discriminator. The generator is composed of a cascaded encoder network and a decoder network. The encoder's input is the SAR image, and the decoder's output is the optical image. The discriminator extracts the texture features of the image and judges the generator's output. The SAR image conversion network constructed in this invention, based on the collaborative work of SAC, FSM, and MCVF modules, can adaptively process complex information in SAR data, exhibits good robustness, can aggregate spatial information, improves the quality and perceptual similarity of SAR image conversion, and significantly improves the output quality of the optical image.

Description

Technical Field

[0001] This invention relates to the field of synthetic aperture radar (SAR) image processing technology, and more particularly to the field of SAR image conversion technology, specifically a method for constructing a SAR image conversion network. Background Technology

[0002] In the field of Earth observation, optical remote sensing images capture solar radiation reflected from ground objects, providing intuitive data that aligns with human visual interpretation habits and is rich in spectral, texture, and color information, greatly facilitating applications such as target identification, ground object classification, and change detection. However, the imaging quality of optical sensors is heavily dependent on sunlight and atmospheric conditions, making them ineffective in darkness or in adverse weather conditions such as clouds, rain, or fog, significantly limiting their continuous observation capabilities. Synthetic Aperture Radar (SAR), as an active microwave remote sensing sensor, images the Earth's surface by emitting electromagnetic waves and receiving their backscattered signals. Its unique imaging mechanism grants it all-weather, all-day operation capabilities, enabling it to penetrate clouds, rain, fog, and certain levels of vegetation and ground surface, acquiring stable and reliable data without relying on sunlight or weather conditions, offering new possibilities for solving the problem of thick clouds.

[0003] Despite the rapid development of image processing methods led by deep learning technology in recent years, existing image processing design and analysis still mainly focus on general natural images, i.e., optical images. Existing image-based data analysis methods, when applying SAR image analysis, require design based on the imaging principles of SAR images, which is costly. Directly applying SAR images for analysis does not achieve the expected results. How to clearly and accurately interpret SAR images quickly and efficiently, and convert them into natural images such as optical images, is a major challenge for current multi-source data analysis. However, the fundamental differences between SAR images and optical images, and between modal data, present a double challenge for image conversion: on the one hand, the side-looking imaging mechanism of SAR causes a systematic shift in its geometric features compared to optical imaging; on the other hand, there is no direct physical correspondence between microwave backscattering coefficients and optical reflectivity.

[0004] Therefore, there is an urgent need for a network and a network construction method that can interpret synthetic aperture radar (SAR) images and convert them into clear optical images. Summary of the Invention

[0005] The purpose of this invention is to provide a method for constructing a synthetic aperture radar (SAR) image conversion network to solve the problem of poor interpretability of SAR images in the prior art.

[0006] A method for constructing a synthetic aperture radar image conversion network, characterized by comprising:

[0007] S1. Construct an encoder network based on the Transformer architecture. The encoder network contains multiple coding layers, which are connected in series. Each coding layer consists of a continuous Transformer module and a frequency-space domain modulation module.

[0008] S2. Construct a decoder network and upsample the final output features of the encoder network through deconvolution. The encoder network contains multiple decoding layers, and each decoding layer corresponds to one of the encoding layers. In each decoding layer, information from the corresponding encoding layer is concatenated through skip connections.

[0009] S3. A synthetic aperture radar image conversion network is constructed based on a generative adversarial network framework, including a generator and a discriminator. The generator is composed of a cascaded encoder network and a decoder network, which is used to convert synthetic aperture radar images to generate optical images. The input of the encoder network is the synthetic aperture radar image, and the output of the decoder network is the optical image. The discriminator extracts the texture features of the image through multi-scale convolutional layers and is used to judge the output of the generator.

[0010] A window partitioning module is set before the first coding layer. The image segmentation size is set in the window partitioning module. The synthetic aperture radar image is divided into multiple fixed-size image blocks by a non-overlapping block through a linear embedding layer. The pixel values ​​of the synthetic aperture radar image are flattened in the channel direction to realize the pixel value connection of the image blocks and obtain the image block features. The image block features are used as the input of the continuous Transformer module of the first coding layer.

[0011] A window merging module is set between two adjacent coding layers. The output feature map of the previous coding layer is processed by the Linear Embedding layer, which halves the resolution of the output feature map and doubles the number of channels of the output feature map to build a feature pyramid and provide input feature maps for the next coding layer.

[0012] The continuous Transformer module is a series structure consisting of a multi-head perceptron module, a layer normalization module, a cognitive vision filter module, and a window attention module. The window attention module includes standard multi-head attention and moving window multi-head attention. The cognitive vision filter includes an input scaling module and a multi-scale visual information integration module.

[0013] The frequency-space domain modulation module decouples the output features of the continuous Transformer module by learning parameters through Fourier transform, and converts the frequency response of the output features of the continuous Transformer module into weight information in the spatial domain to obtain dynamic weights.

[0014] A distribution adaptation layer is set in the input scale adjustment module. By setting the distribution adaptation layer, the continuous Transformer module is improved, and the information capture capability of the continuous Transformer module is enhanced. The multi-scale visual information integration module contains three convolutional filters. All three convolutional filters adopt depthwise separable convolution, and the kernel sizes are 3×3, 5×5 and 7×7, respectively.

[0015] Two learning weights are introduced in the distribution adaptation layer to adjust the input ratio between the module input and the skip connection input of the multi-scale visual information integration module. In the multi-scale visual information integration module, three depthwise separable convolutions with different kernel sizes are used to process the module input. Three feature maps are obtained based on the skip connection input of the multi-scale visual information integration module.

[0016] The average of the three feature maps is calculated, a residual structure is added to the depthwise separable convolution, and the module input of the multi-scale visual information integration module is fused with the average of the three feature maps through residual skip connections to obtain the depthwise convolution features.

[0017] In the multi-scale visual information integration module, point convolution is set, and residual structures are set in the point convolution. The deep convolution features are aggregated by 1×1 point convolution, and the aggregated features are fused with the deep convolution features by skip connections to obtain the output features of the multi-scale visual information integration module.

[0018] The continuous Transformer module uses a multi-head perceptron module as a feedforward neural network, combined with a window attention module and a cognitive visual filter module to process the input features. The feature processing of the consecutive Transformer modules in the encoding layer is as follows:

[0019] ;

[0020] ;

[0021] in, Indicates the first Output features of successive Transformer modules in the layer coding layer Indicates the first The intermediate output features of successive Transformer modules in the encoding layer, when When it is greater than 1, Indicates the first The output features of successive Transformer modules in the layer coding layer, when hour, Represents image block features. This indicates that features are processed using a conventional multi-head attention mechanism. This indicates that features are processed through a cognitive visual filter module. This indicates that the features are subjected to layer normalization. This indicates that a feedforward network is applied to process the features;

[0022] In adjacent coding layers of the encoder network, the conventional multi-head attention mechanism is used alternately, followed by a shifting window multi-head attention mechanism. The feature processing of the encoder's successive Transformer modules in the layer is as follows:

[0023] ;

[0024] ;

[0025] in, No. The intermediate output features of successive Transformer modules in the encoding layer. Indicates the first Output features of consecutive Transformer modules with +1 coding layer This indicates that features are processed through multi-head attention using a moving window.

[0026] A Fourier parameter budget module is set in the frequency-space domain modulation module to set the Fourier parameter budget. The Fourier parameter budget represents the total number of Fourier parameters acquired by the frequency-space domain modulation module. The frequency-space domain modulation module (FSM) extracts the Fourier parameters of the input features of the frequency-space domain modulation module based on the Fourier parameter budget. The Fourier parameters are associated with the frequency coordinates of the Fourier domain, and each parameter corresponds to a specific frequency component. According to the frequency components corresponding to each Fourier parameter, the Fourier parameters are sorted from low frequency to high frequency. The Fourier parameters are transformed to the spatial domain through inverse discrete Fourier transform to obtain the spatial domain features corresponding to each Fourier parameter. The spatial domain features after inverse discrete Fourier transform are jointly modulated with the spatial domain features of the input features of the frequency-space domain modulation module to generate dynamic weights. Based on the dynamic weights, attention matching is performed on the frequency domain of the spatial domain features of the input features of the frequency-space domain modulation module.

[0027] The height of the output feature map of the continuous Transformer module is Width is The features are processed by the frequency domain-spatial domain modulation module to obtain the spatial domain features of each Fourier parameter;

[0028] The obtained Spatial domain characteristics of Fourier parameters for:

[0029] ;

[0030] in, Coordinate values ​​representing the characteristics of the spatial domain. Indicates the first Fourier parameters, This represents the frequency domain coordinates of the corresponding Fourier parameters in the height direction, with values ​​ranging from 0 to... , This represents the frequency coordinate of the Fourier parameters in the width direction, with values ​​ranging from 0 to... , This is a complex exponential term used to convert the frequency domain information of the Fourier parameters to the spatial domain;

[0031] After obtaining the spatial domain features, the spatial domain features are cropped, including setting frequency domain intervals or frequency domain thresholds and cropping the spatial domain features based on the frequency domain intervals, and cropping the spatial domain features using random cropping methods to obtain spatial domain cropped features. These spatial domain cropped features are then divided according to frequency domain information or spatial data. Groups, each group corresponding to a group feature Each group feature corresponds to a weight. Grouping features Global average pooling is performed, followed by feature transformation through a fully connected layer. Finally, the attention coefficients for each group are generated using the sigmoid function. Dynamic weights are calculated based on the attention coefficients and weights of each group. :

[0032] ;

[0033] ;

[0034] in, , Indicates grouping features Perform global average pooling. This indicates that features are processed through a fully connected layer.

[0035] Define the isolation frequency range and frequency band focus, and based on the isolation frequency range, decompose the frequency response of the dynamic weight into... Each frequency band, based on frequency band attention Attention matching is performed on each frequency band to obtain the output results for each frequency band. A modulation map is generated from the output results, and stable optical features are output to obtain the optical image of the synthetic aperture radar image.

[0036] The synthetic aperture radar image conversion network achieves end-to-end image conversion between synthetic aperture radar images and corresponding optical images through adversarial training of the generator and discriminator;

[0037] The generator adopts an encoding and decoding network architecture, consisting of a cascaded encoder network and a decoder network. The encoder network consists of four coding layers, and the decoder network corresponds to the encoder network and consists of four decoding layers. The encoder network uses consecutive Transformer blocks and frequency-spatial domain modulation modules as encoding modules. The synthetic aperture radar image enters the decoder network after passing through the encoder network consisting of four coding layers. The decoder network uses two-dimensional transposed convolution to gradually restore the image size and obtain the output image of the decoder network. The input features of the encoder network are used as information supplements for the decoder network by skip connections.

[0038] The loss function of the synthetic aperture radar image conversion network is used to guide the optimization direction of the generator and the discriminator. It includes the generator loss function and the discriminator loss function. The generator loss function consists of adversarial loss, cycle consistency loss and identity loss function. The discriminator includes two independent sub-discriminators. The discriminator loss function consists of the loss functions of the two sub-discriminators.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] This invention constructs an encoder network based on the Swin Transformer architecture, sets up a continuous Transformer module and a frequency-spatial domain modulation module in the encoder network, and uses the unique moving window mechanism of the continuous Transformer module to sample SAR images layer by layer and extract multi-scale features of SAR images.

[0041] This invention embeds a cognitive visual filter in a continuous Transformer module and sets a distributed adaptation layer in the cognitive filter to adjust the ratio between the module input and the fixed layer input, thereby improving the model's adaptability.

[0042] This invention introduces learnable Fourier parameters into the frequency-space domain modulation module, dynamically and adaptively allocates diverse frequency domain responses for each frequency band, and converts them into weighted information in the spatial domain. This enhances essential features, weakens useless information, and improves the network's ability to process the inherent complex transform domain information and special noise in SAR data, as well as its ability to analyze the unique structural mechanisms and information feature distribution of SAR images.

[0043] This invention significantly improves the performance of the synthetic aperture radar image conversion network by setting up SAC, FSM and MCVF modules in the encoder network and through the synergistic effect between the modules. Attached Figure Description

[0044] Figure 1 This is an overall structural diagram of the synthetic aperture radar image conversion network of the present invention;

[0045] Figure 2 This is a schematic diagram of the continuous Transformer module of the present invention;

[0046] Figure 3 This is a schematic diagram of the frequency-space domain modulation module (FSM) of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] A method for constructing a synthetic aperture radar image conversion network, characterized by comprising:

[0049] S1. An encoder network is constructed based on the Swin Transformer architecture. The encoder network contains multiple coding layers, which are connected in series. Each coding layer consists of a continuous Transformer module and a frequency-space domain modulation module.

[0050] S2. Construct a decoder network and upsample the final output features of the encoder network through deconvolution. The encoder network contains multiple decoding layers, and each decoding layer corresponds to one of the encoding layers. In each decoding layer, information from the corresponding encoding layer is concatenated through skip connections.

[0051] S3. A synthetic aperture radar image conversion network is constructed based on the generative adversarial network framework, including a generator and a discriminator. The generator consists of a cascaded encoder network and a decoder network. The input of the encoder network is the synthetic aperture radar image, and the output of the decoder network is the optical image. The discriminator extracts the texture features of the image through multi-scale convolutional layers and is used to judge the output of the generator.

[0052] The optical image output by the decoder refers to a two-dimensional digital image generated by multimodal feature fusion and intelligent reconstruction of the input SAR image through a synthetic aperture radar image conversion network. This image simulates the imaging effect of an optical sensor in terms of visual performance and is significantly superior to the original SAR image in terms of spatial resolution, spectral features, or semantic information.

[0053] A window partitioning module is set before the first coding layer. The image segmentation size is set in the window partitioning module. The synthetic aperture radar image is divided into multiple fixed-size image blocks by a linear embedding layer. The pixels of the synthetic aperture radar image are flattened in the channel direction to realize the pixel value connection of the image blocks and obtain the image block features. The image block features are used as the input of the continuous Transformer module of the first coding layer. A window merging module is set between two adjacent coding layers. The output feature map of the previous coding layer is processed by the linear embedding layer. The resolution of the output feature map is halved and the channels of the output feature map are doubled to construct a feature pyramid and provide the input feature map for the next coding layer.

[0054] A multi-stage feature encoding network based on continuous Transformer modules is constructed. The input is a SAR image, which is first divided into non-overlapping blocks by a window partitioning module, and then fed into the feature encoding layers of each stage. Each encoding layer consists of two parts: an improved continuous Transformer block and a frequency-spatial domain modulation module. Each encoding layer corresponds to a feature extraction stage. Before the first encoding layer, the SAR image is divided into fixed-size non-overlapping blocks by a window partitioning module. The channel data of each pixel is linearly transformed by a linear embedding layer and then fed into the Transformer module. Starting from the second stage, a window merging operation is performed before each encoding layer to fix the resolution at half of the original and double the depth information. The feature maps processed by the window partitioning module or the window merging module are then fed into the Transformer module.

[0055] The continuous Transformer module uses a multi-head perceptron as a feedforward neural network, combining a multi-head attention mechanism and a cognitive visual filter to process the input features. First, the features of the image after non-overlapping block division are set as the concatenation of the original pixel values, that is, every 4x4 adjacent pixels form an image patch. Then, the image patch is flattened in the channel direction, and the channel data of each pixel is linearly transformed through the Linear Embedding layer and fed into the Transformer module.

[0056] The improved hierarchical architecture of continuous Transformer blocks allows the model to extract features with different resolutions and depths at different stages. Its pyramid-like multi-scale characteristics are attributed to the ingenious downsampling design of Patch Merging. This design uses a 2×2 sliding window to concatenate adjacent pixels of a fixed size along the depth direction. After applying a linear layer, the resolution is finally fixed at half of the original, while doubling the depth information. This is then applied to the Transformer block to maintain the resolution. Patch Merging is a key downsampling operation in hierarchical visual Transformers (such as SwinTransformer) used to reduce feature map resolution and increase channel dimensions between different stages, with the goal of constructing a feature pyramid.

[0057] The unique properties of SAR images, including speckle noise and distinctive texture structures, have always posed challenges to object identification and classification. Furthermore, the high-resolution satellite imagery they belong to often leads to a surge in computational resources for models, making practical deployment difficult. To achieve efficient modeling, conventional windowed multi-head attention (W-MSA) and moving windowed multi-head attention (SW-MSA) are used to replace the computationally complex multi-head attention (MSA), and a multi-cognitive visual filter (MCVF) is designed to improve the perception of two-dimensional visual signals.

[0058] There are multiple encoding layers, the first one... The feature processing of the consecutive Transformer modules in the encoding layer is as follows:

[0059] ;

[0060] ;

[0061] in, Indicates the first Output features of successive Transformer modules in the layer coding layer Indicates the first The intermediate output features of successive Transformer modules in the encoding layer, when When it is greater than 1, Indicates the first The output features of successive Transformer modules in the layer coding layer, when hour, Represents image block features. This indicates that features are processed using a conventional multi-head attention mechanism. This indicates that features are processed using a cognitive visual filter. This indicates that the features are subjected to layer normalization. This indicates that a feedforward network is applied to process the features;

[0062] In adjacent coding layers of the encoder network, the conventional multi-head attention mechanism is used alternately, followed by a shifting window multi-head attention mechanism. Therefore, the... The feature processing of the encoder's successive Transformer modules in the layer is as follows:

[0063] ;

[0064] ;

[0065] in, No. The intermediate output features of successive Transformer modules in the encoding layer. Indicates the first Output features of consecutive Transformer modules with +1 coding layer This indicates that features are processed through multi-head attention using a moving window.

[0066] like Figure 2 As shown, by designing and applying Spatial Aggregation Connectivity (SAC), cross-module residual connections are achieved, promoting more accurate spatial information encoding. The principle of SAC is to improve feature representation by aggregating tensor information in the vertical and horizontal directions, converging pixel-level weights, and utilizing cross-module residual connections to promote more accurate spatial information encoding. First, regarding the... Output features of layer coding layer The input is processed and reshaped. Then, a depthwise separable convolution (DConv) is used to decompose the standard convolution into depthwise and pointwise convolutions to extract spatial features, reducing computational cost and parameter count while preserving feature extraction capabilities. Batch normalization (BN) and the GeLU activation function are then applied to process the features. The GeLU activation function introduces non-linearity into the model, increasing its ability to learn complex features. Average pooling is performed on both the vertical (AvgPool_h) and horizontal (AvgPool_w) directions to aggregate tensor information in the vertical and horizontal directions, converging pixel-level weights and synthesizing the spatial information of the features. A 1x1 convolution is applied to adjust the number of feature channels, fusing feature information. Batch normalization and the GeLU activation function are applied again to maintain feature stability and introduce non-linearity, reshaping the feature map to make it suitable for the next encoding layer. Input features Adding the processed features to form residual connections generates an output feature map. These cross-module residual connections help gradient flow and prevent gradient vanishing.

[0067] The Continuous Transformer module is an improved continuous window attention module, consisting of a cascaded structure of a multi-head perceptron, layer normalization, a cognitive visual filter, and a window attention module, with a built-in residual structure. The window attention module includes standard multi-head attention and moving window multi-head attention. The cognitive visual filter includes an input scaling module and a multi-scale visual information integration module. In the external path, the input features of the Continuous Transformer module are directly merged into the total output of the Continuous Transformer module through spatial aggregation (SAG) residual connections. The Cognitive Visual Filter (MCVF) module further enhances the information capture capability of the Continuous Transformer module by adjusting the ratio of module input to fixed layer input.

[0068] A scaled layer (Scaled LayerNorm) consisting of two learnable weights is set outside the canonical layer of the input scaling module. This scaled layer improves the continuous Transformer module, enhancing its information-grasping capabilities. In terms of cognition, a multi-scale visual information integration module is constructed by introducing multiple convolutional filters into the cognitive visual filter module. This integrates visual signals from different scales, leading to a better understanding of the input image. The multi-scale visual information integration module contains three convolutional filters, all employing depthwise separable convolutions with kernel sizes of 3×3, 5×5, and 7×7. Depthwise separable convolutions (DWConv) are used instead of standard convolutions to minimize the size of additional parameters. Two learnable weights are introduced in the scaled layer to adjust the input ratio between the module input and the skip connection input of the multi-scale visual information integration module. The module input is processed using three depthwise separable convolutions with different kernel sizes, resulting in three feature maps based on the skip connection input of the multi-scale visual information integration module.

[0069] Let the feature map of the input multi-scale visual information integration module be... The feature map is processed through a multi-scale visual information integration module. First, the input feature map is normalized by layer, and then two learnable weights are used. and The normalized result is linearly combined with the original input to retain some information from the original input, ensuring stable model training results.

[0070] ;

[0071] in, This represents the feature map after layer normalization and residual combination; This indicates that the features are being normalized at different levels.

[0072] The average of the three feature maps is calculated, a residual structure is added to the depthwise separable convolution, and the module input of the multi-scale visual information integration module is fused with the average of the three feature maps through residual skip connections to obtain the depthwise convolution features.

[0073] ;

[0074] in, Indicates the first The input characteristics of each filter, Indicates passing through the first The input features are processed by depthwise convolution of each filter. This indicates that the average value is being calculated. This represents the feature map after multi-scale depthwise separable convolution and residual connection, which is equivalent to depthwise convolution features.

[0075] In the multi-scale visual information integration module, point convolutions are set up for feature channel adjustment and aggregation. Residual structures are incorporated within these point convolutions, and residual skip connections are introduced to enhance information transfer and model training performance. 1×1 point convolutions are used to aggregate deep convolutional features, and skip connections are then used to fuse the aggregated features with the deep convolutional features to obtain the output features of the multi-scale visual information integration module. :

[0076] ;

[0077] in, This represents a point convolution with a kernel size of 1×1. It aggregates deep convolution features through 1×1 point convolution, and then fuses the aggregated features with deep convolution features, adjusting the number of channels of the features and aggregating feature information from different channels.

[0078] The output of the Transformer module is input into the Frequency-Spatial-Domain Modulation (FSM) module of the coding layer. This module uses the Inverse Discrete Fourier Transform (IDFT) to transform a learnable Fourier parameter budget into the spatial domain and jointly modulates it with the input spatial domain features, thereby converting it into dynamic weights. These weights then allocate attention to the frequency domain of the input spatial domain features, ultimately resulting in feature optimization in the spatial domain. The Frequency-Spatial-Domain Modulation (FSM) module decouples the output features of the continuous Transformer module through Fourier-learned parameters, converting the frequency response of the continuous Transformer module's output features into spatial domain weight information to obtain dynamic weights.

[0079] This invention abandons the traditional CNN approach of independent processing in the spatial domain and adopts a strategy of decoupling through Fourier domain learnable parameters and adding spatial-frequency dual-dimensional modulation. A Fourier parameter budget module is set in the frequency-space domain modulation module to set the Fourier parameter budget. The Fourier parameter budget represents the total number of Fourier parameters acquired by the frequency-space domain modulation module. The frequency-space domain modulation module (FSM) extracts the Fourier parameters of the input features of the frequency-space domain modulation module based on the Fourier parameter budget. The Fourier parameters are associated with the frequency coordinates of the Fourier domain, and each parameter corresponds to a specific frequency component. According to the frequency components corresponding to each Fourier parameter, the Fourier parameters are sorted from low frequency to high frequency. The Fourier parameters are transformed to the spatial domain through inverse discrete Fourier transform to obtain the spatial domain features corresponding to each Fourier parameter. The spatial domain features after inverse discrete Fourier transform are jointly modulated with the spatial domain features of the input features of the frequency-space domain modulation module to generate dynamic weights. Based on the dynamic weights, attention matching is performed on the frequency domain of the spatial domain features of the input features of the frequency-space domain modulation module.

[0080] The height of the output feature map of the continuous Transformer module is Width is The features are processed by the frequency domain-spatial domain modulation module to obtain the spatial domain features of each Fourier parameter;

[0081] The obtained Spatial domain characteristics of Fourier parameters for:

[0082] ;

[0083] in, Coordinate values ​​representing the characteristics of the spatial domain. Indicates the first Fourier parameters, This represents the frequency domain coordinates of the corresponding Fourier parameters in the height direction, with values ​​ranging from 0 to... , This represents the frequency coordinate of the Fourier parameters in the width direction, with values ​​ranging from 0 to... , This is a complex exponential term used to convert the frequency domain information of the Fourier parameters to the spatial domain.

[0084] After obtaining the spatial domain features, the spatial domain features are cropped, including setting frequency domain intervals or frequency domain thresholds and cropping the spatial domain features based on the frequency domain intervals, and cropping the spatial domain features using random cropping methods to obtain spatial domain cropped features. These spatial domain cropped features are then divided according to frequency domain information or spatial data. Groups, each group corresponding to a group feature Each group feature corresponds to a weight. Grouping features Global average pooling is performed, followed by feature transformation through a fully connected layer. Finally, the attention coefficients for each group are generated using the sigmoid function. Dynamic weights are calculated based on the attention coefficients and weights of each group. :

[0085] ;

[0086] ;

[0087] in, , Indicates grouping features Perform global average pooling. This indicates that features are processed through a fully connected layer.

[0088] After pruning and recombination, multiple weights with complementary frequency domain characteristics are obtained. Each weight has a different frequency response, ensuring diverse construction and adaptive adjustment based on input samples. This advantage is attributed to dynamically generated attention coefficients, which are obtained by global average pooling of the input and then passing it through a fully connected FC layer.

[0089] Define the isolation frequency range and frequency band focus, and based on the isolation frequency range, decompose the frequency response of the dynamic weight into... Each frequency band, based on frequency band attention Attention matching is performed on each frequency band to obtain the output results for each frequency band. A modulation map is generated from the output results, and stable optical features are output to obtain the optical image of the synthetic aperture radar image.

[0090] Regarding local information, a lightweight 1D convolution is employed, which accurately captures local channel information while maintaining the ability to learn fine-grained modulation for each element in the weights. To meet the requirements of spatial dynamic modulation, this invention decomposes the frequency response of the obtained dynamic adaptive weights into multiple frequency bands and applies a set of binary masks to isolate specific frequency ranges. The attention to specific frequencies in this invention is as follows:

[0091] ;

[0092] in, For the first The output results of each frequency band in the spatial domain, IDFT stands for Inverse Discrete Fourier Transform, used to convert frequency domain signals back to the spatial domain. Indicates the first A binary mask, used to select frequencies, is a binary vector or matrix. The dot product operator represents the element-wise multiplication of two vectors or matrices. The Discrete Fourier Transform (DFT) is used to convert spatial domain signals to the frequency domain. It represents a spatial domain signal.

[0093] After obtaining the output results for each frequency band, in order to further refine the control of the influence of each frequency band at each spatial location, a modulation map is generated using a standard convolutional layer and a sigmoid activation function. The formula for calculating the output feature map Y is:

[0094] ;

[0095] in, The weights, calculated by performing a 1×1 convolution operation on the input data followed by a Sigmoid activation function, are used to control the contribution of each frequency band to the final output. This invention achieves adaptive multi-band processing by acquiring the output results of each frequency band, providing more precise control over the complex frequency domain information of SAR data.

[0096] The synthetic aperture radar (SAR) image conversion network achieves end-to-end image conversion between SAR images and corresponding optical images through adversarial training between the generator and the discriminator. The generator adopts an encoder-decoder network architecture, consisting of a cascaded encoder network and a decoder network. The encoder network consists of four coding layers, and the decoder network, corresponding to the encoder network, consists of four decoding layers. The encoder network uses consecutive Transformer blocks and a frequency-spatial domain modulation module as its coding module. After passing through the encoder network consisting of four coding layers, the SAR image enters the decoder network. The decoder network uses two-dimensional transposed convolution to gradually restore the image size, obtaining the decoder network output image. The input features of the encoder network are used as information supplements for the decoder network through skip connections. The discriminator extracts image texture features through multi-scale convolutional layers.

[0097] The decoder network contains transposed convolutions, which gradually upsample the lowest-level features of the encoder network through deconvolution to gradually restore the image size. In each decoding layer, information from the encoding layer is spliced ​​together through skip connections, reducing the semantic information differences between deep and shallow layers.

[0098] The loss function of the synthetic aperture radar image conversion network is used to guide the optimization direction of the generator and the discriminator. It includes the generator loss function and the discriminator loss function. The generator loss function consists of adversarial loss, cycle consistency loss and identity loss function. The discriminator includes two independent sub-discriminators. The discriminator loss function is determined by the loss function calculated by the sub-discriminators.

[0099] The generator is used to convert the image between the frequency domain and the spatial domain, and the discriminator is used to determine the authenticity of the frequency domain and spatial domain generated by the generator.

[0100] The objective function of the generator is:

[0101] ;

[0102] in, This represents sample data in the synthetic aperture radar domain. Represents sample data of the target optical domain. and Represents a generator. Sample data from the synthetic aperture radar (SAR) domain is converted to the target optical domain for generating optical images based on SAR images. The sample data of the target optical domain is converted to the synthetic aperture radar domain, which is used to generate synthetic aperture radar images based on the optical images. This represents synthetic aperture radar data that has undergone two transformations. This represents the target optical image data after two conversions. This represents synthetic aperture radar data after one transformation. This represents the target optical image data after one conversion.

[0103] The generator's final loss function consists of adversarial loss, cycle consistency loss, and identity loss function. (Generator loss function) for:

[0104] ;

[0105] in, Represents generator Compared to the discriminator The adversarial loss reflects the generator The generated image can fool the discriminator. ability, Represents generator Compared to the discriminator The adversarial loss, generated by network adversarial training, This represents the cycle consistency loss of the generator. This represents identity loss. Cyclic consistency loss encourages consistency in bidirectional transformations by requiring the result of transformation from one domain to another and back to be as close as possible to the original input, helping to preserve important structural information. Identity loss uses the L1 norm as a measurement standard to ensure that the generated optical image matches the original SAR image in terms of structural and stylistic features.

[0106] The decision maker consists of two independent decision makers. and , Used to determine the authenticity of synthetic spatial radar domain image data generated by the generator. The discriminator is used to determine the authenticity of the image data in the target optical domain generated by the generator. The output of the discriminator is the probability that the image data generated by the generator is judged as real data, with a value range of [0, 1]. In adversarial training, the generator attempts to generate realistic fake data to deceive the discriminator, while the discriminator attempts to accurately distinguish between real data and fake data generated by the generator. Through this adversarial training, the generator can gradually generate more realistic data, and the discriminator can continuously improve its discrimination ability, ultimately making the generated data very close to the real data in terms of statistical characteristics.

[0107] like Figure 1 As shown, the synthetic aperture radar image conversion network uses synthetic aperture radar images as real data input to the generator. The generator's encoding and decoding layers convert the synthetic aperture radar images to generate optical images, including reconstructed optical image data and generated optical image data. The generated optical image data is then input to the discriminator and the generator for the next adversarial training. Through adversarial training between the generator and the discriminator, end-to-end image conversion between synthetic aperture radar images and corresponding optical images is achieved.

[0108] Discriminator loss function The sum of the loss functions of the two sub-discriminators:

[0109] ;

[0110] in, Discriminator loss function, Discriminator The loss function.

[0111] In the patch partitioning module, the image is divided into non-overlapping blocks, and the resulting feature image patches are input into the Linear Embedding layer. After flattening the image patches along the channel direction, the feature maps are input into the continuous Transformer module. The encoder network includes four encoding layers, each corresponding to a feature processing stage. In each encoding layer, spatial aggregation connections are used to fuse the input features of each encoding layer into the output features of the encoding layer. From stage one to stage four, the feature resolution is halved with each stage, equivalent to halving the height and width of the feature map and doubling the channel information dimension, maintaining the same resolution when applied to the continuous Transformer blocks. In this embodiment, stage one (first encoding layer), stage two (second encoding layer), and stage four (fourth encoding layer) each contain two continuous Transformer modules, and stage three (third encoding layer) contains six continuous Transformer modules, which facilitates the extraction and analysis of deep information. The decoder network contains transposed convolutions, which gradually upsample the lowest-level features of the encoder network through deconvolution to gradually restore the image size. In each decoding layer, information from the encoding layer is spliced ​​together through skip connections, reducing the semantic information differences between deep and shallow layers.

[0112] like Figure 2 As shown, the Continuous Transformer module is an improved continuous window attention module, consisting of a cascaded structure of a multi-head perceptron (MLP) feedforward neural network, layer normalization, a cognitive visual filter, and a window attention module, with a built-in residual structure. The window attention module includes both conventional window MLP and moving window MLP, while the cognitive visual filter (MCVF) includes an input scaling module and a multi-scale visual information integration module. In the external path, the input features of the Continuous Transformer module are directly merged into the total output of the Continuous Transformer module through residual connections in the Spatial Aggregation Module (SAG). The cognitive visual filter module further enhances the information capture capability of the Continuous Transformer module by adjusting the ratio of module input to fixed layer input.

[0113] like Figure 3As shown, the output of the continuous Transformer module is input into the Frequency-Spatial-Domain Modulation (FSM) module of the coding layer. Learnable Fourier parameters are introduced into the FSM module. The FSM module uses Inverse Discrete Fourier Transform (iDFT) to transform these learnable Fourier parameters into the spatial domain, and then jointly modulates them with the input spatial domain features through a fully connected activation layer. This transforms the input spatial domain features into a dynamic adaptive weight, which performs attention decomposition on the frequency domain of the input spatial domain features, ultimately converting it into feature optimization in the spatial domain.

[0114] To ensure that the network architecture constructed in this invention is superior to the baseline model, comparative experiments were conducted on the model. The performance of different translation methods on different datasets was compared and analyzed. The results are shown in Tables 1-2.

[0115] Dataset description:

[0116] The SEN1-2 dataset covers the globe and all weather seasons, containing 282,384 pairs of synthetic aperture radar / optical image pairs with a resolution of 10 meters from Sentinel-1 and Sentinel-2 satellites. Each image is 256×256 pixels in size. Each image patch (picture block) in the dataset has undergone rigorous quality control to ensure accurate alignment between the SAR and optical images. This invention uses data from the SEN1-2 dataset, dividing the data into training and test sets at a 5:1 ratio.

[0117] The QXS-SAROPT dataset contains 20,000 pairs of high-resolution (1-meter) synthetic aperture radar (SAR) and optical satellite image patches, featuring rich scene features such as roads, buildings, and ships. Each image is 256×256 pixels in size. This invention uses data from the QXS-SAROPT dataset and divides the data into training and test sets at a 5:1 ratio.

[0118] This invention validates the effectiveness of the proposed method through three parts: dataset description, comparative experiments, and ablation experiments. All experiments were conducted using the PyTorch framework, with an experimental environment including NVIDIA and Linux operating systems. To optimize the network, the Adam optimizer was selected, with the two exponential decay factors β1 and β2 set to 0.5 and 0.999, respectively. A linear learning rate decay strategy was employed, reducing the learning rate by linearly varying the multiplication factor. The initial learning rate for the first 100 epochs was 0.0002 (training epochs were set to 200). There was one input channel and three output channels. The network generated in the last epoch served as the final network, the Synthetic Aperture Radar Image Conversion Network (DFA-former), used for testing and validation.

[0119] Translation methods include Pix2pix, CycleGAN, MUNIT, UGATIT, Parallel-GAN, WFLM-GAN, MT-GAN, and DFA-former. Pix2pix is ​​an image-to-image translation method based on generative adversarial networks (GANs). It aims to solve a range of image translation problems in supervised learning, using traditional UNET and ResNet as network architectures. CycleGAN is a deep learning method for unsupervised image-to-image translation. Unlike Pix2Pix, CycleGAN can perform image translation without matching paired training data, making it more flexible and practical in many real-world applications. MUNIT is an unsupervised image generation method that introduces the concept of content and style separation for stylistically diverse outputs. UGATIT proposes an unsupervised image-to-image translation method combining novel attention and normalization functions. Parallel-GAN proposes a SAR-to-optical image translation method based on hierarchical latent features, promoting clear output through a two-stage translation and reconstruction process. WFLM-GAN utilizes a multi-scale GAN based on wavelet feature learning to translate SAR images into optical images. MT-GAN improves the translation network's ability to parse SAR images by preprocessing them. The translated image serves as a reference for cloud removal in the optical image.

[0120] Tables 1 and 2 present the evaluation metrics results of image translation comparison experiments using different translation methods on different datasets. Performance evaluations of different image translation methods were conducted, including Peak Signal-to-Noise Ratio (PSNR) in dB, which measures the similarity between the reconstructed and original images (higher values ​​indicate better image quality); Structural Similarity (SSIM), ranging from [0, 1], which measures the structural similarity between two images (values ​​closer to 1 indicate greater similarity); Correlation Coefficient (CC), ranging from [0, 1], which measures the linear correlation between two images (values ​​closer to 1 indicate stronger correlation); Dissimilarity Coefficient (KID), which measures the difference between the distribution of the generated image and the distribution of the real image (lower values ​​indicate a closer resemblance between the generated and real image distributions); and Perceptual Similarity Index (LPIPS), ranging from [0, 1], where lower values ​​indicate greater perceptual similarity between the two images.

[0121] Table 1 Comparison Experiment of Image Translation on SEN1-2 Dataset

[0122] .

[0123] As shown in Table 1, in the SEN1-2 dataset, the synthetic aperture radar image conversion network provided by this invention achieved excellent results in all five evaluation metrics, indicating that the network takes into account both global and detailed aspects during the synthetic aperture radar image translation process. In terms of peak signal-to-noise ratio (PSNR), the network model provided by this invention is 4 dB higher than the traditional supervised Pix2pix and approximately 2.3 dB higher than the unsupervised image conversion model CycleGAN. In terms of SSIM, the network model provided by this invention is approximately 0.44 and 0.35 higher, respectively, demonstrating that the network structure provided by this invention is superior in feature and structure reconstruction.

[0124] MUNIT and UGATIT, as representatives of image transformation and translation in the field of natural imagery, possess highly efficient inter-domain transformation models. However, the remote sensing imagery domain is characterized by complex scenes, diverse data structures, and numerous defects such as speckle noise (small targets, low resolution, etc.), making them difficult to adapt to complex and ever-changing new tasks. Parallel-GAN employs a two-stage process, translating first and then enhancing, significantly improving the quality of the translated image across various metrics. However, its enhancement stage only targets existing optical features and cannot compensate for features missing in the translation result itself. The WFLM-GAN method utilizes wavelet transform to perform in-depth decomposition of SAR images, mitigating the unique noise inherent in SAR. However, its performance is limited by its cumbersome multi-feature processing and traditional network kernel. MT-GAN uses a similar processing procedure to parallel-GAN for preprocessing SAR data, but its performance in color and texture restoration is somewhat lacking. Compared with MT-GAN, the network model provided in this invention achieves an overall performance improvement of over 15%.

[0125] Table 2 Comparison Experiment of Image Translation on QXS-SAROPT Dataset

[0126] .

[0127] As shown in Table 2, the translation network proposed in this invention also achieved superior results across various metrics in the QXS-SAROPT dataset. Compared to the method with the second-best translation performance, the method of this invention improved the PSNR metric by approximately 1.64 dB, the SSIM metric by approximately 0.06, and the CC metric by 0.1223. It also reduced the KID and LPIPS metrics by 0.0167 and 0.053, respectively. This demonstrates that the translation conversion of this invention has significant advantages not only in the conversion of macroscopic and microscopic information but also in perceptual reconstruction. The QXS-SAROPT dataset contains a rich variety of scenes and numerous small targets, including high-resolution scenes such as residential buildings, factories, amusement parks, and docks, as well as minute targets such as vehicles, ships, and roads, posing a significant challenge to the matching analysis of SAR and optical data. Previous image translation methods often struggle to maintain the structural integrity and texture realism of targets when handling such complex scenes, resulting in insufficient performance limits and problems such as blurring, distortion, or semantic misalignment in the generated results. Traditional CycleGAN and Pix2pix, due to their monotonous network structures, have insufficient performance limits when dealing with complex scenes and small targets, making it difficult to extract deep semantics and resulting in distortion and blurring in the translated images. Other methods have improved model complexity to some extent, but they struggle to adapt to the complex internal information and unique speckle noise of SAR data, so the translated images all exhibit varying degrees of object loss and feature distortion.

[0128] This invention conducts ablation experiments on different combinations of network modules on the SEN1-2 dataset, and the experimental results are shown in Table 3.

[0129] Table 3 Ablation experiments of different parts of the network in the SEN1-2 dataset.

[0130] .

[0131] Table 3 presents the ablation experiment results for each component of the network provided in this invention, demonstrating the effectiveness of each part. The traditional Swin-Transformer module's encoding / decoding network was used as the baseline network, and each structure was fused for analysis. Evaluation metrics included PSNR (Peak Signal-to-Noise Ratio, higher is better), SSIM (Structural Similarity, higher is better), CC (Correlation Coefficient, higher is better), KID (Kernel Inception Distance, lower is better), and LPIPS (Learned Perceptual Image Patch Similarity, lower is better). Table 3 shows that each sub-module has varying degrees of effectiveness. Specifically, combining with the SAC module slightly improves overall performance. The SAC module enhances spatial information through aggregation, improving global modeling capabilities. The FSM module improves PSNR and SSIM by approximately 3% and 8%, respectively. As the core processing module for SAR features, it uses Fourier domain parameter decoupling and spatial-frequency dual-dimensional modulation to achieve truly segmented adaptive processing of dynamic weights, effectively handling the complex spatial and frequency domain relationships of SAR data. The learnable scaling factor in the MCVF module, along with the multi-receptive-field separation convolutional coding and averaging step, better integrates global and local information appropriately. It also exhibits high robustness to occlusion and noise without significantly increasing computational burden, effectively improving the image quality of the model output. As shown in Table 3, the complete model DFA-former achieves the best results across all metrics, indicating that the combination of SAC, FSM, and MCVF modules works synergistically to significantly improve model performance. Specifically, the SAC module improves image quality and perceptual similarity. The FSM module excels in feature selection and optimization, particularly in improving PSNR and LPIPS. Tables 1-3 demonstrate that the synthetic aperture radar image conversion network DFA-former provided by this invention achieves the best performance.

[0132] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing a synthetic aperture radar image conversion network, characterized in that, include: S1. Construct an encoder network based on the Transformer architecture. The encoder network contains multiple coding layers, which are connected in series. Each coding layer consists of a continuous Transformer module and a frequency-space domain modulation module. S2. Construct a decoder network and upsample the final output features of the encoder network through deconvolution. The encoder network contains multiple decoding layers, and each decoding layer corresponds to one of the encoding layers. In each decoding layer, information from the corresponding encoding layer is concatenated through skip connections. S3. A synthetic aperture radar image conversion network is constructed based on the generative adversarial network framework, including a generator and a discriminator. The generator is composed of a cascaded encoder network and a decoder network, which is used to convert synthetic aperture radar images to generate optical images. The input of the encoder network is the synthetic aperture radar image, and the output of the decoder network is the optical image. The discriminator extracts the texture features of the image entering the discriminator through multi-scale convolutional layers, which are used to judge the output of the generator. The continuous Transformer module is a series structure consisting of a multi-head perceptron module, a layer normalization module, a cognitive vision filter module, and a window attention module. The window attention module includes standard multi-head attention and moving window multi-head attention. The cognitive vision filter module includes an input scaling module and a multi-scale visual information integration module. The frequency-space domain modulation module decouples the output features of the continuous Transformer module by learning parameters through Fourier transform, and converts the frequency response of the output features of the continuous Transformer module into weight information in the spatial domain to obtain dynamic weights. A distribution adaptation layer is set in the input scale adjustment module. By setting the distribution adaptation layer, the continuous Transformer module is improved, and the information capture capability of the continuous Transformer module is enhanced. The multi-scale visual information integration module contains three convolutional filters. All three convolutional filters adopt depthwise separable convolution, and the kernel sizes are 3×3, 5×5 and 7×7, respectively. Two learning weights are introduced in the distribution adaptation layer to adjust the input ratio between the module input and the skip connection input of the multi-scale visual information integration module. In the multi-scale visual information integration module, three depthwise separable convolutions with different kernel sizes are used to process the module input. Three feature maps are obtained based on the skip connection input of the multi-scale visual information integration module. The average of the three feature maps is calculated, a residual structure is added to the depthwise separable convolution, and the module input of the multi-scale visual information integration module is fused with the average of the three feature maps through residual skip connections to obtain the depthwise convolution features. In the multi-scale visual information integration module, point convolution is set, and residual structures are set in the point convolution. The deep convolution features are aggregated by 1×1 point convolution, and the aggregated features are fused with the deep convolution features by skip connections to obtain the output features of the multi-scale visual information integration module.

2. The method for constructing a synthetic aperture radar image conversion network according to claim 1, characterized in that, A window partitioning module is set before the first coding layer. The image segmentation size is set in the window partitioning module. The synthetic aperture radar image is divided into multiple fixed-size image blocks by a non-overlapping block through a linear embedding layer. The pixel values ​​of the synthetic aperture radar image are flattened in the channel direction to realize the pixel value connection of the image blocks and obtain the image block features. The image block features are used as the input of the continuous Transformer module of the first coding layer. A window merging module is set between two adjacent coding layers. The output feature map of the previous coding layer is processed by the Linear Embedding layer, which halves the resolution of the output feature map and doubles the number of channels of the output feature map to build a feature pyramid and provide the input feature map for the next coding layer.

3. The method for constructing a synthetic aperture radar image conversion network according to claim 2, characterized in that, The continuous Transformer module uses a multi-head perceptron module as a feedforward neural network, combined with a window attention module and a cognitive visual filter module to process the input features. The feature processing of the consecutive Transformer modules in the encoding layer is as follows: ； ； in, Indicates the first Output features of successive Transformer modules in the layer coding layer Indicates the first The intermediate output features of successive Transformer modules in the encoding layer, when When it is greater than 1, Indicates the first The output features of successive Transformer modules in the layer coding layer, when hour, Represents image block features. This indicates that features are processed using a conventional multi-head attention mechanism. This indicates that features are processed through a cognitive visual filter module. This indicates that the features are subjected to layer normalization. This indicates that a feedforward network is applied to process the features; In adjacent coding layers of the encoder network, the conventional multi-head attention mechanism is used alternately with the moving window multi-head attention mechanism. No. The feature processing of the encoder's successive Transformer modules in the layer is as follows: ； ； in, No. The intermediate output features of successive Transformer modules in the encoding layer. Indicates the first Output features of consecutive Transformer modules with +1 coding layer This indicates that features are processed through multi-head attention using a moving window.

4. The method for constructing a synthetic aperture radar image conversion network according to claim 3, characterized in that, A Fourier parameter budget module is set in the frequency-space domain modulation module to set the Fourier parameter budget. The Fourier parameter budget represents the total number of Fourier parameters acquired by the frequency-space domain modulation module. The frequency-space domain modulation module (FSM) extracts the Fourier parameters of the input features of the frequency-space domain modulation module based on the Fourier parameter budget. The Fourier parameters are associated with the frequency coordinates of the Fourier domain, and each parameter corresponds to a specific frequency component. According to the frequency components corresponding to each Fourier parameter, the Fourier parameters are sorted from low frequency to high frequency. The Fourier parameters are transformed to the spatial domain through inverse discrete Fourier transform to obtain the spatial domain features corresponding to each Fourier parameter. The spatial domain features after inverse discrete Fourier transform are jointly modulated with the spatial domain features of the input features of the frequency-space domain modulation module to generate dynamic weights. Based on the dynamic weights, attention matching is performed on the frequency domain of the spatial domain features of the input features of the frequency-space domain modulation module.

5. The method for constructing a synthetic aperture radar image conversion network according to claim 4, characterized in that, The height of the output feature map of the continuous Transformer module is Width is The features are processed by the frequency domain-spatial domain modulation module to obtain the spatial domain features of each Fourier parameter; The obtained Spatial domain characteristics of each Fourier parameter for: ； in, Coordinate values ​​representing the characteristics of the spatial domain. Indicates the first Fourier parameters, This represents the frequency domain coordinates of the corresponding Fourier parameters in the height direction, with values ​​ranging from 0 to... , This represents the frequency coordinate of the Fourier parameters in the width direction, with values ​​ranging from 0 to... , This is a complex exponential term used to convert the frequency domain information of the Fourier parameters to the spatial domain; After obtaining the spatial domain features, the spatial domain features are cropped, including setting frequency domain intervals or frequency domain thresholds and cropping the spatial domain features based on the frequency domain intervals, and cropping the spatial domain features using random cropping methods to obtain spatial domain cropped features. These spatial domain cropped features are then divided according to frequency domain information or spatial data. Groups, each group corresponding to a group feature Each group feature corresponds to a weight. Grouping features Global average pooling is performed, followed by feature transformation through a fully connected layer. Finally, the attention coefficients for each group are generated using the sigmoid function. Dynamic weights are calculated based on the attention coefficients and weights of each group. : ； ； in, , Indicates grouping features Perform global average pooling. This indicates that features are processed through a fully connected layer.

6. The method for constructing a synthetic aperture radar image conversion network according to claim 5, characterized in that, Define the isolation frequency range and frequency band focus, and based on the isolation frequency range, decompose the frequency response of the dynamic weight into... Each frequency band, based on frequency band attention Attention matching is performed on each frequency band to obtain the output results for each frequency band. A modulation map is generated from the output results, and stable optical features are output to obtain the optical image of the synthetic aperture radar image.

7. The method for constructing a synthetic aperture radar image conversion network according to claim 6, characterized in that, The synthetic aperture radar image conversion network achieves end-to-end image conversion between synthetic aperture radar images and corresponding optical images through adversarial training of the generator and discriminator; The generator adopts an encoding and decoding network architecture, consisting of a cascaded encoder network and a decoder network. The encoder network consists of four coding layers, and the decoder network corresponds to the encoder network and consists of four decoding layers. The encoder network uses consecutive Transformer blocks and frequency-spatial domain modulation modules as encoding modules. The synthetic aperture radar image enters the decoder network after passing through the encoder network consisting of four coding layers. The decoder network uses two-dimensional transposed convolution to gradually restore the image size and obtain the output image of the decoder network. The input features of the encoder network are used as information supplements for the decoder network by skip connections.

8. The method for constructing a synthetic aperture radar image conversion network according to claim 7, characterized in that, The loss function of the synthetic aperture radar image conversion network is used to guide the optimization direction of the generator and the discriminator. It includes the generator loss function and the discriminator loss function. The generator loss function consists of adversarial loss, cycle consistency loss and identity loss function. The discriminator includes two independent sub-discriminators. The discriminator loss function consists of the loss functions of the two sub-discriminators.

Images