Interference protection template generation method for conversion from optical image to SAR (Synthetic Aperture Radar) image

Abstract

The invention provides an interference protection template generation method for conversion from an optical image to an SAR image, and relates to the technical field of light-SAR image conversion, and the method comprises the steps: obtaining a to-be-converted optical image of a target region; inputting the optical image to be converted into a pre-training SAR image generator, and converting to obtain a target SAR image; the target SAR image is used as an interference protection template to be sent to an interference target party; the pre-training SAR image generator is trained by adopting a CycleGAN architecture, and the pre-training SAR image generator is a network model fusing a Pixel ViT structure and a UNet structure; the overall loss function of the CycleGAN architecture is constructed by adopting perception loss, pixel loss and cyclic consistency loss; a discriminator in the CycleGAN architecture is provided with a batch head module. On the basis, the generation quality and effect of the interference protection template are integrally improved, and the acquisition cost of the interference protection template is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of optical-SAR image conversion, and in particular to a method for generating an interference protection template for converting an optical image into a SAR image. Background Art

[0002] As an advanced microwave remote sensing radar technology, Synthetic Aperture Radar (SAR) plays a vital role in many fields such as geological mapping, target monitoring, and key area reconnaissance with its all-day, all-weather high-resolution imaging capabilities. However, with the widespread application of SAR technology, its powerful imaging capabilities have gradually become a potential threat to malicious reconnaissance. In order to effectively counter the enemy's malicious reconnaissance using SAR and protect the safety of important targets or areas, SAR interference protection technology has emerged. The core of this technology is to generate high-quality, high-fidelity interference protection templates, but traditional SAR interference protection template generation methods have disadvantages such as high cost, complex processes or high geometric modeling requirements, and high computing power requirements, which are difficult to meet the needs of actual applications.

[0003] With the rapid development of artificial intelligence (AI) technology, deep learning-based methods for generating SAR interference protection templates have gradually become a research hotspot. Among them, the interference protection template generation method based on CycleGAN (Cycle-Consistent Generative Adversarial Networks) introduces a cycle consistency loss to achieve unsupervised bidirectional conversion between optical and SAR images, eliminating the need for paired training data and effectively addressing the scarcity of paired training data in cross-modal image conversion. This method has been widely used in the field of SAR image generation and provides a new approach for generating high-quality SAR interference protection templates. CycleGAN achieves image style transfer and cross-modal conversion through adversarial training of the generator and discriminator, as well as the constraint of the cycle consistency loss.

[0004] Despite CycleGAN's impressive performance in cross-modal image translation, it still suffers from significant deficiencies in the optical-to-SAR conversion process. Due to a lack of strictly paired training data, CycleGAN primarily relies on a cycle consistency loss to constrain network training. This relatively weak constraint makes the generated SAR images susceptible to distortion that does not conform to the target domain's characteristics. Specifically, the lack of a strongly constrained loss function (such as a pixel-level loss or feature matching loss) makes the network prone to conversion errors during training, resulting in differences in texture, structure, and other aspects from the real SAR images. This problem is particularly prominent in the optical-to-SAR conversion task, as the imaging principles and representations of the two modalities differ significantly, making it easy for the network to incur deviations during the conversion process. Furthermore, directly generating SAR interference templates faces the challenge of acquiring SAR images. SAR image acquisition requires specialized SAR systems, which are expensive, and the imaging process is susceptible to weather, time, and other constraints, making it difficult to ensure timely acquisition of the required image data. Consequently, existing technologies suffer from poor SAR interference protection template generation and high generation costs. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a method for generating an interference protection template for converting an optical image into a SAR image.

[0006] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] The present invention provides a method for generating an interference protection template for converting an optical image into a SAR image, comprising:

[0008] acquiring an optical image of a target area to be converted;

[0009] The optical image to be converted is input into the pre-trained SAR image generator to obtain the target SAR image;

[0010] The target SAR image is sent to the interference target party as an interference protection template; the interference target party is the party that conducts unauthorized reconnaissance of the target area; the pre-trained SAR image generator is trained using the CycleGAN architecture, and the pre-trained SAR image generator is a network model that integrates the Pixel ViT and UNet structures; the overall loss function of the CycleGAN architecture is constructed using perceptual loss, pixel loss and cycle consistency loss; the discriminator in the CycleGAN architecture is equipped with a batch head module, and the discriminator uses a feature cache mechanism to perform distribution statistics of image features.

[0011] Optionally, the pre-trained SAR image generator includes: a Pixel ViT model, an encoder with a UNet structure, and a decoder with a UNet structure;

[0012] The Pixel ViT model is set between the encoder of the UNet structure and the decoder of the UNet structure. The Pixel ViT model, the encoder of the UNet structure and the decoder of the UNet structure together constitute a U-shaped network structure.

[0013] Optionally, the encoder of the UNet structure is provided with a plurality of encoder layers connected in sequence; the decoder of the UNet structure is provided with a plurality of decoder layers connected in sequence; the number of encoder layers is equal to the number of decoder layers, and a one-to-one correspondence is formed on both sides of the U-shaped network structure;

[0014] Each encoder layer is provided with a convolution module and a downsampling module connected in sequence;

[0015] Each decoder layer is provided with an upsampling module and a convolution module connected in sequence;

[0016] The output end of each up-sampling module is also connected to the input end of the down-sampling module on the corresponding side of the U-shaped network structure.

[0017] Optionally, the convolution module is provided with a first convolution layer, a first instance normalization layer, a first activation layer, a second convolution layer, a second instance normalization layer and a second activation layer connected in sequence;

[0018] The first activation layer and the second activation layer both use the Leaky ReLU activation function.

[0019] Optionally, the discriminator includes: a plurality of sequentially connected discriminative convolutional layers, a first-in-first-out buffer module, and a batch head module;

[0020] The batch head module includes a batch normalization layer, a third convolutional layer, a ReLu activation layer, and a fourth convolutional layer connected in sequence;

[0021] The FIFO cache module is used to store the features of real optical sample images, the features of real SAR image samples, the features of SAR generated images, and the features of optically generated images; the FIFO cache module uses a FIFO strategy to update the stored features.

[0022] Optionally, the CycleGAN architecture is provided with: a pre-trained SAR image generator, a pre-trained optical image generator, a pre-trained SAR image discriminator, and a pre-trained optical image discriminator;

[0023] The pre-trained SAR image discriminator and the pre-trained optical image discriminator have the same structure and are both trained using Wasserstein loss optimization with gradient penalty.

[0024] The pre-trained SAR image generator and the pre-trained optical image generator have the same structure and are jointly optimized using perceptual loss, pixel loss, cycle consistency loss, and Wasserstein loss with gradient penalty.

[0025] Alternatively, the overall loss function is expressed as:

[0026]

[0027] in, Represents the initial SAR image generator G O2S , initial optical image generator G S2O , initial SAR image discriminator D SAR and the initial optical image discriminator D OPT The value of the overall loss function obtained is, represents the initial SAR image generator G O2S The value of the total loss function, Denotes the initial optical image generator G S2O The value of the total loss function, Denotes the initial optical image discriminator D OPT The corresponding value of Wasserstein loss with gradient penalty, Denotes the initial SAR image discriminator D SAR The corresponding value of Wasserstein loss with gradient penalty.

[0028] Optionally, the initial SAR image generator G O2S The total loss function is expressed as:

[0029]

[0030] represents the initial SAR image generator G O2S The corresponding value of Wasserstein loss with gradient penalty, represents the initial SAR image generator G O2S The corresponding pixel loss value, represents the initial SAR image generator G O2S The corresponding perceptual loss value, represents the initial SAR image generator G O2S The corresponding cycle consistency loss value; λ pix represents the pixel loss weight of the initial SAR image generator, λ per represents the perceptual loss weight of the initial SAR image generator, λ cycrepresents the cycle consistency loss weight of the initial SAR image generator;

[0031]

[0032]

[0033] Where x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) The result of image discrimination, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the sampled x and y, y represents the real SAR sample image, ||·||1 represents the L1 norm, ||·||2 represents the L2 norm, Indicates G O2S (x) Pre-trained VGG16 network The size of the j-th layer is C j ×H j ×W j The feature map, C j 、H j 、W j Represents G O2S (x) After The number of channels, width and height of the feature map corresponding to the j-th layer feature extraction; Indicates that y passes through The feature map generated by the jth layer, N represents the total number of layers of the pre-trained VGG16 network, G S2O (G O2S (x)) represents the G O2S (x) Input the initial optical image generator G S2O The resulting optically generated image, Indicates G O2S (x) After The feature map produced by the j-th layer.

[0034] Optionally, the initial SAR image discriminator D SAR The corresponding Wasserstein loss with gradient penalty is expressed as:

[0035]

[0036] x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) is the result of image discrimination, y is the real SAR sample image, D SAR (y) represents the initial SAR image discriminator D SAR The result of image discrimination on y, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the obtained x and y, λ GP Represents the hyperparameter corresponding to the balanced gradient penalty term and Wasserstein distance, represents the SAR generated image, represents the distribution of all SAR generated images, Indicates from Sampling obtained And calculate the expectation of the expression in (·); Indicates the use of the initial SAR image discriminator D SAR right The result of image discrimination is Denotes the initial SAR image discriminator D SAR about gradient.

[0037] Optionally, the training process of the pre-trained SAR image generator includes:

[0038] Acquire an image sample set; the image sample set includes a plurality of real optical sample images and real SAR sample images corresponding to the real optical sample images;

[0039] Input the image sample set into the CycleGAN architecture and train it in the direction of decreasing the value of the overall loss function;

[0040] The initial SAR image generator in the CycleGAN architecture that meets the iteration stopping condition is used as the pre-trained SAR image generator;

[0041] The conditions for stopping the iteration include: the number of iterations meets the iteration threshold or the value of the overall loss function is continuously less than the loss threshold;

[0042] The initial SAR image generator has the same structure as the pre-trained SAR image generator and is the pre-trained SAR image generator in the training process.

[0043] The present invention provides a method for generating an interference protection template for converting an optical image into a SAR image, comprising the following steps: obtaining an optical image to be converted of a target area; inputting the optical image to be converted into a pre-trained SAR image generator to convert the image into a target SAR image; and sending the target SAR image as an interference protection template to an interference target party, wherein the interference target party is a party that conducts unauthorized reconnaissance of the target area; the pre-trained SAR image generator is trained using a CycleGAN architecture, and the pre-trained SAR image generator is a network model that integrates a Pixel ViT and a UNet structure; an overall loss function of the CycleGAN architecture is constructed using perceptual loss, pixel loss, and cycle consistency loss; a discriminator in the CycleGAN architecture is provided with a batch head module, and the discriminator uses a feature cache mechanism to perform distribution statistics of image features. In this paper, by integrating PixelViT and UNet structures, the ability of the pre-trained SAR image generator to capture long-range dependencies and retain local details is improved, and the generation quality of the interference protection template is improved to a certain extent; secondly, the overall loss function of perceptual loss, pixel loss and cycle consistency is proposed. Perceptual loss ensures that the SAR generated image is consistent with the real SAR sample image at the semantic level, pixel loss constrains the similarity between the SAR generated image and the real SAR sample image at the pixel level, and cycle consistency loss ensures the consistency of the overall content between the input sample image and the SAR generated image, thereby improving the SAR image generation effect of the pre-trained SAR image generator as a whole; finally, by setting a batch head module in the discriminator, the statistical distribution of the training sample image can be accurately captured, and the ability to distinguish the diversity of the generated images is enhanced. In summary, through the setting of the network structure, the setting of the overall loss function and the setting of the batch head module, the generation quality and effect of the interference protection template are improved as a whole. Moreover, since the interference protection template is converted from the optical image, the problem of difficulty in obtaining SAR images is avoided, and the acquisition cost of the interference protection template is reduced.

[0044] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic flow chart of a method for generating an interference protection template for converting an optical image to a SAR image provided by an embodiment of the present invention;

[0046] Figure 2 The overall conversion architecture diagram during the initial SAR image generator training process is shown as an example;

[0047] Figure 3 The schematic diagram of the structure of the pre-trained SAR image generator is shown as an example;

[0048] Figure 4 The structural diagram of the discriminator is shown as an example;

[0049] Figure 5 A comparison diagram of the visual effects of SAR-generated images based on the method of the present invention is shown as an example. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0051] In order to improve the generation quality and effect of interference protection templates and reduce the acquisition cost of interference protection templates, an embodiment of the present invention provides an interference protection template generation method for converting optical images to SAR images. Figure 1 A flow chart of a method for generating an interference protection template for converting an optical image to a SAR image provided by an embodiment of the present invention is shown as follows: Figure 1 Shown, including:

[0052] S101 , obtaining an optical image of a target area to be converted.

[0053] S102: Input the optical image to be converted into a pre-trained SAR image generator, and convert it to obtain a target SAR image.

[0054] It should be noted that the pre-trained SAR image generator is trained using the CycleGAN architecture, and the pre-trained SAR image generator is a network model that integrates Pixel ViT and UNet structures; the overall loss function of the CycleGAN architecture is constructed using perceptual loss, pixel loss, and cycle consistency loss; the discriminator in the CycleGAN architecture is equipped with a batch head module, and the discriminator uses a feature cache mechanism to perform distribution statistics of image features.

[0055] Figure 2 The overall conversion architecture diagram during the training process of the initial SAR image generator is shown as an example. The initial SAR image generator is a pre-trained SAR image generator during the training process. Figure 2As shown, the optical sample image is input into the initial SAR image generator to obtain the SAR generated image. The difference between the SAR generated image and the SAR sample image matched with the optical sample image is calculated through the WGAN-GP loss (Wasserstein loss with gradient penalty) + pixel loss + perceptual loss + cycle consistency loss, and the discriminator D SAR The difference between the SAR generated image and the SAR sample image is calculated, and finally the pre-trained SAR image generator is obtained through the overall loss function.

[0056] Figure 3 The schematic diagram of the structure of the pre-trained SAR image generator is shown as an example. Figure 3 As shown in Figure 1, the pre-trained SAR image generator includes: Pixel ViT model, UNet structure encoder and UNet structure decoder;

[0057] The Pixel ViT model is set between the encoder of the UNet structure and the decoder of the UNet structure. The Pixel ViT model, the encoder of the UNet structure and the decoder of the UNet structure together constitute a U-shaped network structure.

[0058] Optionally, the encoder of the UNet structure is provided with a plurality of encoder layers connected in sequence; the decoder of the UNet structure is provided with a plurality of decoder layers connected in sequence; the number of encoder layers is equal to the number of decoder layers, and a one-to-one correspondence is formed on both sides of the U-shaped network structure;

[0059] Each encoder layer is provided with a convolution module and a downsampling module connected in sequence;

[0060] Each decoder layer is provided with an upsampling module and a convolution module connected in sequence;

[0061] The output end of each up-sampling module is also connected to the input end of the down-sampling module on the corresponding side of the U-shaped network structure.

[0062] The embodiment of the present invention is exemplified by assuming that there are four encoder layers:

[0063] In the first encoder layer, after downsampling, the image width and height are halved, while the number of feature channels remains unchanged. The output dimensions are (w,h,f) = (w0 / 2,h0 / 2,f0). Specifically, the convolutional layer of the convolutional module has a kernel size of 3×3, a stride of 1, and padding of 1. The convolutional layer of the downsampling module has a kernel size of 2×2, a stride of 2, and padding of 0. w0, h0, and f0 are the width and height of the input image and the number of feature channels in the first encoder layer, respectively.

[0064] In the second, third, and fourth encoder layers, the input is the output of the previous encoder layer. After downsampling, the width and height of the output image are halved, while the number of feature channels is doubled. The output dimensions of the fourth downsampling module are (w0 / 16, h0 / 16, 8f0). Specifically, the convolutional layer of the convolutional module has a kernel size of 3×3, a stride of 1, and padding of 1. The convolutional layer of the downsampling module has a kernel size of 2×2, a stride of 2, and padding of 0.

[0065] Due to the inherent shortcomings of the convolutional layer, the encoder layer will cause the loss of some image information during the process of downsampling the input image. Therefore, a jump connection is also set between the encoder layer and the decoder layer (the output end of the upsampling module is also connected to the input end of the downsampling module on the corresponding side of the U-shaped network structure). Through the jump connection of the corresponding encoder layer and decoder layer, some information can be directly transferred to the decoder layer. The present invention helps to generate a more refined reconstructed image by adding a jump connection. The upsampling module improves the spatial resolution of the feature map through bilinear interpolation and achieves efficient upsampling through sub-pixel convolution.

[0066] Since four encoder layers are used for this example, four corresponding decoder layers are also set. That is, after the output of the Pixel ViT passes through the upsampling module of the first decoder layer, it is spliced ​​along the channel with the output of the convolution module of the fourth encoder layer, and then input into the convolution module of the first encoder layer. Specifically, the convolution kernel size of the convolution layer of the convolution module is 3×3, the stride is 1, and the padding is 1. The convolution kernel size of the convolution layer of the upsampling module is also 3×3, the stride is 1, and the padding is 1.

[0067] In the second, third, and fourth decoder layers, the input to the upsampling module is the output of the previous decoder layer. The output of the upsampling module is concatenated with the output of the convolution module of the corresponding encoder layer along the channel and then input into the decoder's convolution module. The second, third, and fourth decoder layers correspond to the third, second, and first encoder layers, respectively. Specifically, the convolution kernel size of the convolution module convolution layer is 3×3, with a stride of 1 and padding of 1. The convolution kernel size of the convolution layer of the upsampling module is 3×3, with a stride of 1 and padding of 1.

[0068] In addition, it should be noted that the number of Transformer encoder modules in the Pixel ViT in the embodiment of the present invention is specifically 10. The specific value is determined through experiments, and can be referred to the subsequent experimental section for description.

[0069] Optionally, the convolution module is provided with a first convolution layer, a first instance normalization layer, a first activation layer, a second convolution layer, a second instance normalization layer and a second activation layer connected in sequence;

[0070] The first activation layer and the second activation layer both use the Leaky ReLU activation function.

[0071] Figure 4 The schematic diagram of the structure of the discriminator is shown as an example. Figure 4 As shown in Figure 1, the discriminator includes: multiple sequentially connected discriminative convolutional layers (i.e., the main discriminator PatchGAN removes the last layer), a first-in-first-out cache module, and a batch head module;

[0072] The batch head module includes a batch normalization layer, a third convolutional layer, a ReLu activation layer, and a fourth convolutional layer connected in sequence;

[0073] The FIFO cache module is used to store the features of real optical sample images, the features of real SAR image samples, the features of SAR generated images, and the features of optically generated images; the FIFO cache module uses a FIFO strategy to update the stored features.

[0074] It should be noted that the embodiments of the present invention introduce a composite discriminator architecture that combines the advantages of a main discriminator and a batch head. The main discriminator uses the PatchGAN architecture, which divides the image into multiple small patches and independently discriminates each patch. This approach improves the discriminator's attention to local image details, placing greater emphasis on local image features when generating high-quality images. In this way, PatchGAN can effectively capture subtle differences in images and enhance the model's generative capabilities.

[0075] The batch header module is used to extract batch statistical features. It consists of a sequentially connected batch normalization layer, a third convolutional layer, a Relu activation layer, and a fourth convolutional layer. This effectively integrates global statistical features at the batch level, enhancing the model's ability to capture global image information. To accommodate the complex data characteristics of optical-to-SAR image conversion, the batch header module employs a cache mechanism specifically designed to store batch features. The cached data features are divided into four categories: features of real optical sample images, features of real SAR image samples, features of SAR-generated images, and features of optically generated images. These caches have a fixed size and are updated using a first-in-first-out policy. This cache mechanism dynamically captures the statistical distribution of current training samples while preserving historical features, better adapting to the diversity of image generation tasks. By introducing this composite discriminator architecture, the model balances local and global features while improving the quality and diversity of generated images. Particularly for optical-to-SAR image conversion, the batch header's cache mechanism effectively promotes the diversity of generated images and enhances the overall discriminative power of the discriminator.

[0076] Optionally, the CycleGAN architecture is provided with: a pre-trained SAR image generator, a pre-trained optical image generator, a pre-trained SAR image discriminator, and a pre-trained optical image discriminator;

[0077] The pre-trained SAR image discriminator and the pre-trained optical image discriminator have the same structure and are both trained using Wasserstein loss optimization with gradient penalty.

[0078] The pre-trained SAR image generator and the pre-trained optical image generator have the same structure and are jointly optimized using perceptual loss, pixel loss, cycle consistency loss, and Wasserstein loss with gradient penalty.

[0079] Alternatively, the overall loss function is expressed as:

[0080]

[0081] in, Represents the initial SAR image generator G O2S , initial optical image generator G S2O , initial SAR image discriminator D SAR and the initial optical image discriminator D OPT The value of the overall loss function obtained is, represents the initial SAR image generator G O2S The value of the total loss function, Denotes the initial optical image generator G S2O The value of the total loss function, Denotes the initial optical image discriminator D OPT The corresponding value of Wasserstein loss with gradient penalty, Denotes the initial SAR image discriminator D SAR The corresponding value of Wasserstein loss with gradient penalty.

[0082] Optionally, the initial SAR image generator G O2S The total loss function is expressed as:

[0083]

[0084] represents the initial SAR image generator G O2S The corresponding value of Wasserstein loss with gradient penalty, represents the initial SAR image generator G O2S The corresponding pixel loss value, represents the initial SAR image generator G O2S The corresponding perceptual loss value, represents the initial SAR image generator G O2S The corresponding cycle consistency loss value; λ pixrepresents the pixel loss weight of the initial SAR image generator, λ per represents the perceptual loss weight of the initial SAR image generator, λ cyc represents the cycle consistency loss weight of the initial SAR image generator;

[0085]

[0086] Where x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) The result of image discrimination, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the sampled x and y, y represents the real SAR sample image, ||·||1 represents the L1 norm, ||·||2 represents the L2 norm, Indicates G O2S (x) Pre-trained VGG16 network The size of the j-th layer is C j ×H j ×W j The feature map, C j 、H j 、W j Represents G O2S (x) After The number of channels, width and height of the feature map corresponding to the j-th layer feature extraction; Indicates that y passes through The feature map generated by the jth layer, N represents the total number of layers of the pre-trained VGG16 network, G S2O (G O2S (x)) represents the G O2S (x) Input the initial optical image generator G S2O The resulting optically generated image, Indicates G O2S (x) After The feature map produced by the j-th layer.

[0087] It should be noted that N specifically represents the total number of specific convolutional layers selected for feature extraction in the pre-trained VGG16 network. The selection of specific convolutional layers can be flexibly selected according to actual conditions and is not limited in this embodiment.

[0088] Optionally, the initial SAR image discriminator D SAR The corresponding Wasserstein loss with gradient penalty is expressed as:

[0089]

[0090] x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) is the result of image discrimination, y is the real SAR sample image, D SAR (y) represents the initial SAR image discriminator D SAR The result of image discrimination on y, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the obtained x and y, λ GP Represents the hyperparameter corresponding to the balanced gradient penalty term and Wasserstein distance, represents the SAR generated image, represents the distribution of all SAR generated images, Indicates from Sampling obtained And calculate the expectation of the expression in (·); Indicates the use of the initial SAR image discriminator D SAR right The result of image discrimination is Denotes the initial SAR image discriminator D SAR about gradient.

[0091] Since the pre-trained SAR image discriminator and the pre-trained optical image discriminator have the same structure, the pre-trained SAR image generator and the pre-trained optical image generator have the same structure. Therefore, similarly, the pre-trained optical image generator G can be obtained. S2O The loss function and pre-trained optical image discriminator D OPT The loss function

[0092]

[0093] Denotes the initial optical image generator G S2O The value of the total loss function, Denotes the initial optical image generator G S2O The corresponding value of Wasserstein loss with gradient penalty, Denotes the initial optical image generator G S2O The corresponding pixel loss value, Denotes the initial optical image generator G S2O The corresponding perceptual loss value, Denotes the initial optical image generator G S2O The corresponding cycle consistency loss value; λ pix ' represents the pixel loss weight of the initial optical image generator, λ per ' represents the perceptual loss weight of the initial optical image generator, λ cyc ' represents the cycle consistency loss weight of the initial optical image generator.

[0094] Denotes the initial optical image discriminator D OPT The corresponding Wasserstein loss value with gradient penalty, x represents the real optical sample image, y represents the real SAR sample image, G S2O (y) represents the input of the real SAR sample image y into the initial optical image generator G S2O Get the optically generated image, D OPT (G S2O (y)) represents the initial optical image discriminator D OPT To G S2O (y) The result of image discrimination, D OPT (x) represents the initial optical image discriminator D OPT The result of image discrimination on x, E x～X (·) represents sampling x from the real optical sample image distribution X and using x to calculate the expectation of the expression in (·); λ GP Represents the hyperparameter corresponding to the balanced gradient penalty term and Wasserstein distance, represents the optically generated image, represents the distribution of all optically generated images, Indicates from Sampling obtained And calculate the expectation of the expression in (·); Denotes the use of the initial optical image discriminator D OPT right The result of image discrimination is Denotes the initial optical image discriminator D OPT about The gradient of E y～Y (·) represents sampling y from the real SAR sample image distribution Y and using y to calculate the expectation of the expression in (·).

[0095] as well as The specific expression can refer to as well as This will not be described in detail in this embodiment.

[0096] Optionally, the training process of the pre-trained SAR image generator includes:

[0097] Acquire an image sample set; the image sample set includes a plurality of real optical sample images and real SAR sample images corresponding to the real optical sample images;

[0098] Input the image sample set into the CycleGAN architecture and train it in the direction of decreasing the value of the overall loss function;

[0099] The initial SAR image generator in the CycleGAN architecture that meets the iteration stopping condition is used as the pre-trained SAR image generator;

[0100] The conditions for stopping the iteration include: the number of iterations meets the iteration threshold or the value of the overall loss function is continuously less than the loss threshold;

[0101] The initial SAR image generator has the same structure as the pre-trained SAR image generator and is the pre-trained SAR image generator in the training process.

[0102] S103: Send the target SAR image as an interference protection template to the interference target party.

[0103] It should be noted that the interference target party is the party that conducts unauthorized reconnaissance of the target area. Interference protection templates are generally divided into: target deception interference templates and scene deception interference templates. The target SAR image in the embodiment of the present invention can generally be sent to the interference target party as a scene deception interference template.

[0104] Furthermore, the scene deception jamming template digitally represents the desired jamming protection scene, where the pixels reflect the spatial position and backscatter coefficient of each component within the scene. Therefore, based on the spatial position and backscatter coefficient of each scattering element in the scene deception jamming template, the intercepted enemy SAR signal can be subjected to corresponding time delay and amplitude modulation, ultimately forming a jamming protection signal. This jamming protection signal is then transmitted to the target party, thereby protecting against unauthorized reconnaissance.

[0105] Here, the interference protection signal can be generated using existing methods. For specific generation methods, please refer to "Research on a False Large-Scene SAR Fast Forwarding Deception Interference Method".

[0106] The embodiment of the present invention provides a method for generating interference protection templates for optical image to SAR image conversion. By integrating Pixel ViT and UNet structures, the ability of the pre-trained SAR image generator to capture long-range dependencies and retain local details is improved, thereby improving the generation quality of the interference protection template to a certain extent. Secondly, an overall loss function of perceptual loss, pixel loss, and cycle consistency is proposed. The perceptual loss ensures that the SAR generated image is consistent with the real SAR sample image at the semantic level. The pixel loss constrains the similarity between the SAR generated image and the real SAR sample image at the pixel level. The cycle consistency loss ensures the consistency of the overall content between the input sample image and the SAR generated image, thereby improving the SAR image generation effect of the pre-trained SAR image generator as a whole. Finally, by setting a batch head module in the discriminator, the statistical distribution of the training sample images can be accurately captured, thereby enhancing the ability to distinguish the diversity of the generated images. In summary, through the setting of the network structure, the setting of the overall loss function, and the setting of the batch head module, the generation quality and effect of the interference protection template are improved as a whole. Moreover, since the interference protection template is converted from the optical image, the problem of difficulty in obtaining SAR images is avoided, and the acquisition cost of the interference protection template is reduced.

[0107] In order to illustrate the effectiveness of the interference protection template generation method for converting optical images to SAR images provided by the embodiment of the present invention, a simulation experiment was also conducted as follows:

[0108] The experimental environment and parameter settings are as follows: All experiments are implemented on the deep learning framework PyTorch and trained and evaluated on NVIDIA RTX4090.

[0109] During the training process, the Adam optimizer was used to optimize the network parameters, with β1 = 0.5 and β2 = 0.99. In the experiment, the batch size was set to 1 and the training was carried out for 200 cycles. The learning rate of the generator was set to 1×10 -4, the learning rate of the discriminator is set to 5×10 -5 The present invention sets the weight parameter λ of the cycle consistency loss cyc Set to 10, λ GP Set to 10, and Set to 25, λ per Set to 0.2.

[0110] Among objective evaluation metrics, root mean square error (RMSE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) are the most common technical standards for evaluating image quality. The experimental process uses these objective evaluation metrics to evaluate the image quality of the proposed method.

[0111] The hyperparameter experiments are as follows:

[0112] (1) In order to explore the effect of the cache mechanism of the batch head module in the composite discriminator in the conversion of optical images to SAR images and to explore the influence of cache capacity, three different discriminator architectures were designed: a baseline model containing only PatchGAN, a batch head of PatchGAN+batch head size=3, and a batch head of PatchGAN+batch head size=6. To ensure the comparability and reliability of the experiments, except for the different discriminator architectures, the rest of the training settings were consistent with CTE-GAN (the method of the present invention). The experimental results are shown in Table 1. The experimental results show that after the introduction of the batch head, the quality of the generated image is improved, indicating that the cache mechanism effectively enhances the visual fidelity of the image by fusing global statistical features. However, the larger the cache capacity, the better. The experiment shows that the image generation quality is best when the cache capacity is 3, indicating that a balance should be achieved between historical feature memory and computational efficiency.

[0113] Table 1 Image quality evaluation results under different discriminators

[0114]

[0115]

[0116] (2) The impact of the number of Transformer encoder layers on the quality of generated images. In order to explore the impact of the number of Transformer encoder layers on the quality of generated images, the present invention conducted an ablation experiment in Pixel ViT and set different numbers of Transformer encoder layers. They are 6 layers, 8 layers, 10 layers and 12 layers respectively, to analyze the impact of the change in the number of layers on the generation effect. To ensure the comparability and reliability of the experiment, except for the number of encoder layers, the rest of the training settings are consistent with the CTE-GAN (method of the present invention) settings. The experimental results are shown in Table 2. The results show that when 10 layers of Transformer encoder modules are stacked in Pixel ViT, the PSNR and SSIM of the generated image are both optimal, indicating that a good balance has been achieved between global modeling capabilities and local detail reconstruction.

[0117] Table 2 Image quality evaluation results under different Transformer encoder layers

[0118]

[0119] The ablation experiment is as follows:

[0120] In the method of the present invention, both pixel loss and perceptual loss play a unique role in improving the quality of generated images. To explore the contribution of pixel loss and perceptual loss in generating SAR scene interference protection templates, the performance of four groups of models was compared. The baseline network removed pixel loss and perceptual loss. The baseline network + pixel loss model retained the pixel loss and deleted the perceptual loss. The baseline network + perceptual loss model deleted the pixel loss and retained only the perceptual loss. The rest of the training details were consistent with CTE-GAN. The experiments were conducted on the same training and test sets, and the quantitative indicators obtained are shown in Table 3. The experimental results show that the full model outperforms other models in terms of PSNR and SSIM on all test sets, while also having a lower RMSE, indicating that the full model can achieve superior generation quality in both structural and visual effects when generating SAR scene interference protection templates. In addition, adding pixel loss and perceptual loss to the baseline network (the baseline network is the method of the present invention without pixel loss and perceptual loss) can significantly improve the PSNR and SSIM of the generated images, further improving the detail quality and overall consistency of the generated images.

[0121] Table 3. Image quality evaluation results of ablation experiments

[0122]

[0123] Experimental results show that CTE-GAN (the method of the present invention) outperforms other models in all indicators, indicating that the combination of pixel loss and perceptual loss can effectively improve the generation quality of SAR scene interference protection templates. Compared with the baseline network, the model with pixel loss added alone has a slight improvement in PSNR and SSIM, while the RMSE is reduced, indicating that pixel loss can reduce pixel-level errors and make the generated image closer to the real SAR image in low-level details. The model with perceptual loss added alone has a more obvious improvement in PSNR and RMSE compared to the baseline network, but the SSIM is slightly reduced, indicating that perceptual loss helps to enhance the overall structural consistency of the image, but may not effectively optimize local pixel relationships. Under the joint action of pixel loss and perceptual loss, the complete CTE-GAN model has the highest PSNR and SSIM and the lowest RMSE, indicating that the combination of the two can achieve the best balance between detail fidelity and global structural consistency, so that the generated image is significantly improved in both structure and visual quality. Figure 5 The following is an exemplary diagram showing the visual effect comparison of SAR-generated images based on the method of the present invention. Figure 5 Figure (a) is the input optical image, Figure 5 Figure (b) is a SAR image generated based on the method of the present invention. Figure 5 Figure (c) is the real SAR image (i.e. the SAR image that is expected to be generated). Figure 5 It can be seen that the SAR image generated based on the method of the present invention is similar to the real SAR image in details, which proves the effectiveness of the method of the present invention.

[0124] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic devices can be desktop computers, portable computers, smart mobile terminals, servers, etc., which are not limited in the embodiment of the present invention.

[0125] In the description of this specification, the reference terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0126] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the above-mentioned disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the word "comprising" does not exclude other components or steps, "one" or "an" does not exclude multiple situations, and the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0127] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention cannot be considered to be limited to these descriptions. For those skilled in the art of the present invention, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for generating an interference protection template for converting an optical image to a SAR image, characterized in that: include: acquiring an optical image of a target area to be converted; Inputting the optical image to be converted into a pre-trained SAR image generator to convert the image into a target SAR image; The target SAR image is sent to the interference target party as an interference protection template; the interference target party is the party that conducts unauthorized reconnaissance of the target area; the pre-trained SAR image generator is trained using the CycleGAN architecture, and the pre-trained SAR image generator is a network model that integrates the Pixel ViT and UNet structures; the overall loss function of the CycleGAN architecture is constructed using perceptual loss, pixel loss, and cycle consistency loss; the discriminator in the CycleGAN architecture is provided with a batch head module, and the discriminator uses a feature cache mechanism to perform distribution statistics of image features.

2. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 1, wherein: The pre-trained SAR image generator includes: a Pixel ViT model, an encoder with a UNet structure, and a decoder with a UNet structure; The Pixel ViT model is arranged between the encoder of the UNet structure and the decoder of the UNet structure. The Pixel ViT model, the encoder of the UNet structure and the decoder of the UNet structure together constitute a U-shaped network structure.

3. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 2, wherein: The encoder of the UNet structure is provided with a plurality of encoder layers connected in sequence; the decoder of the UNet structure is provided with a plurality of decoder layers connected in sequence; the number of the encoder layers is equal to the number of the decoder layers, and a one-to-one correspondence is formed on both sides of the U-shaped network structure; Each encoder layer is provided with a convolution module and a downsampling module connected in sequence; Each of the decoder layers is provided with an upsampling module and a convolution module connected in sequence; The output end of each up-sampling module is also connected to the input end of the down-sampling module on the corresponding side of the U-shaped network structure.

4. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 3, wherein: The convolution module is provided with a first convolution layer, a first instance normalization layer, a first activation layer, a second convolution layer, a second instance normalization layer and a second activation layer connected in sequence; The first activation layer and the second activation layer both use a Leaky ReLU activation function.

5. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 1, wherein: The discriminator includes: a plurality of sequentially connected discriminant convolutional layers, a first-in-first-out cache module and the batch head module; The batch head module includes a batch normalization layer, a third convolutional layer, a ReLu activation layer, and a fourth convolutional layer connected in sequence; The first-in-first-out cache module is used to store the features of real optical sample images, the features of real SAR image samples, the features of SAR generated images, and the features of optically generated images; the first-in-first-out cache module adopts a first-in-first-out strategy to update the stored features.

6. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 1, wherein: The CycleGAN architecture is provided with: a pre-trained SAR image generator, a pre-trained optical image generator, a pre-trained SAR image discriminator, and a pre-trained optical image discriminator; The pre-trained SAR image discriminator and the pre-trained optical image discriminator have the same structure, and are both trained using Wasserstein loss optimization with gradient penalty; The pre-trained SAR image generator and the pre-trained optical image generator have the same structure and are obtained by jointly optimizing perceptual loss, pixel loss, cycle consistency loss, and Wasserstein loss with gradient penalty.

7. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 1, wherein: The overall loss function is expressed as: in, Represents the initial SAR image generator G O2S , initial optical image generator G S2O , initial SAR image discriminator D SAR and the initial optical image discriminator D OPT The value of the overall loss function obtained is, represents the initial SAR image generator G O2S The value of the total loss function, Denotes the initial optical image generator G S2O The value of the total loss function, Denotes the initial optical image discriminator D OPT The corresponding value of Wasserstein loss with gradient penalty, Denotes the initial SAR image discriminator D SAR The corresponding value of Wasserstein loss with gradient penalty.

8. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 7, wherein: Initial SAR image generator G O2S The total loss function is expressed as: represents the initial SAR image generator G O2S The corresponding value of Wasserstein loss with gradient penalty, represents the initial SAR image generator G O2S The corresponding pixel loss value, represents the initial SAR image generator G O2S The corresponding perceptual loss value, represents the initial SAR image generator G O2S The corresponding cycle consistency loss value; λ pix represents the pixel loss weight of the initial SAR image generator, λ per represents the perceptual loss weight of the initial SAR image generator, λ cyc represents the cycle consistency loss weight of the initial SAR image generator; Where x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) The result of image discrimination, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the sampled x and y, y represents the real SAR sample image, ||·||1 represents the L1 norm, ||·||2 represents the L2 norm, Represents G O2S (x) Pre-trained VGG16 network The size of the j-th layer is C j ×H j ×W j The feature map, C j 、H j 、W j Represents G O2S (x) After The number of channels, width and height of the feature map corresponding to the j-th layer feature extraction; Indicates that y passes through The feature map generated by the jth layer, N represents the total number of layers of the pre-trained VGG16 network, G S2O (G O2S (x)) represents the G O2S (x) Input the initial optical image generator G S2O The resulting optically generated image, Indicates G O2S (x) After The feature map produced by the j-th layer.

9. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 7, wherein: The initial SAR image discriminator D SAR The corresponding Wasserstein loss with gradient penalty is expressed as: x represents the real optical sample image, G O2S (x) represents the input of x into the initial SAR image generator G O2S The obtained SAR generated image, D SAR (G O2S (x)) represents the initial SAR image discriminator D SAR To G O2S (x) is the result of image discrimination, y is the real SAR sample image, D SAR (y) represents the initial SAR image discriminator D SAR The result of image discrimination on y, E x～X (·) represents sampling x from the real optical sample image distribution X and calculating the expectation of the expression in (·); E x～X,y～Y (·) represents sampling x from the real optical sample image distribution X and obtaining y corresponding to x from the real SAR sample image distribution Y, and calculating the expectation of the expression (·) based on the obtained x and y, λ GP Represents the hyperparameter corresponding to the balanced gradient penalty term and Wasserstein distance, represents the SAR generated image, represents the distribution of all SAR generated images, Indicates from Sampling obtained And calculate the expectation of the expression in (·); Indicates the use of the initial SAR image discriminator D SAR right The result of image discrimination is Denotes the initial SAR image discriminator D SAR about gradient.

10. The method for generating an interference protection template for converting an optical image to a SAR image according to claim 1, wherein: The training process of the pre-trained SAR image generator includes: Acquire an image sample set; the image sample set is provided with a plurality of real optical sample images and real SAR sample images corresponding to the real optical sample images; Inputting the image sample set into the CycleGAN architecture and training in a direction in which the value of the overall loss function decreases; The initial SAR image generator in the CycleGAN architecture that meets the iteration stopping condition is used as the pre-trained SAR image generator; The iteration stopping condition includes: the number of iterations meets the iteration number threshold or the value of the overall loss function is continuously less than the loss threshold; The initial SAR image generator has the same structure as the pre-trained SAR image generator and is the pre-trained SAR image generator in the training process.

Images