SAR-optical image translation method and system based on image evaluation and feature selection

Abstract

The present invention discloses a SAR optical image translation method and system based on image evaluation and feature selection. The method includes establishing a residual mapping network based on the geometric distortion and low-resolution characteristics of SAR images in combination with feature selection; establishing an image generation network based on local information, global information, and information fusion characteristics; designing a loss constraint function based on SAR image generative adversarial, image denoising, and evaluation guidance characteristics; and designing a multi-scale decision network based on the loss constraint algorithm. The residual mapping network and the image generation network are combined to establish an optical image translation model. During the feature extraction process, more attention is paid to important information in SAR images that has a strong correspondence with optical images, which can better achieve consistency in information representation between optical and SAR sensors. The image translation model algorithm proposed in the present invention achieves better translation results in both single-scene specific translation tasks and mixed-scene complex translation tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of SAR-optical image translation, and in particular to a SAR-optical image translation method and system based on image evaluation and feature selection. Background Art

[0002] As the wave of deep learning sweeps across various fields, more and more researchers are adopting neural networks to process SAR images, and new attempts are constantly being made in the field of SAR-to-optical image translation. Existing image translation models primarily rely on generative adversarial networks. The generator in the network is responsible for encoding and extracting features from the input SAR image, converting and decoding it into the corresponding optical image. The discriminator is responsible for detecting the generated image as a forgery and using the judgment results to guide the generator's work, achieving an optimal balance between the two through game theory. In the image translation process, how to optimally extract and filter features determines the ultimate performance of the translation model.

[0003] The significant difference between the imaging mechanism of SAR images, which is based on signal processing algorithms, and the time-integrated exposure imaging method of optical images results in completely different noise characteristics for the two images. Researchers at home and abroad have successively achieved translation of paired and unpaired images using network models such as pix2pix, CycleGAN, and pix2pixHD. However, in this process, researchers have overlooked the specific processing of SAR image noise, resulting in significant noise interference in the generated optical images and poor translation results. Furthermore, most existing image translation models are limited to mapping between optical images, leaving SAR image feature extraction, noise screening, and preservation of image semantic information as pressing challenges. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a SAR-optical image translation method and system based on image evaluation and feature selection, which can solve the problems of feature extraction, noise screening and preservation of image semantic information of SAR images.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: a SAR-optical image translation method based on image evaluation and feature selection, comprising:

[0008] According to the geometric distortion and low-resolution characteristics of SAR images, a residual mapping network is established in combination with feature selection;

[0009] Combining local information, global information and information fusion characteristics, an image generation network is established;

[0010] Design loss constraint functions based on SAR image generation adversarial, image denoising, and evaluation guidance features;

[0011] According to the loss constraint algorithm, a multi-scale decision network is designed, and the residual mapping network is combined with the image generation network to establish an optical image translation model.

[0012] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, wherein: the multi-scale decision network includes:

[0013]

[0014] Among them, λ DCT Represents the weight value of the discrete cosine loss function, λ LPIPS represents the weight value of the perceptual distance loss function based on deep texture structure, λ MSE Represents the weight value of the mean square error loss function based on pixel calibration, L cGAN (G,D k ) represents the loss function based on generative adversarial, L DCT (G,D k ) represents the loss function based on image denoising, L MSE (G,D k ) is the mean square error loss function based on pixel calibration, L LPIPS (G,D k ) is the perceptual distance loss function based on deep texture structure, G * represents the generation network, k is 1-2, representing different discriminant networks, D * Represents the discriminant network, D1 is the first discriminant network, and D2 is the second discriminant network.

[0015] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, the error loss function based on mean square error statistics includes:

[0016]

[0017] Among them, X and Y represent the SAR image and optical image input to the network respectively, h and w represent the length and width of the image matrix respectively, i and j represent each pixel unit in the horizontal and vertical directions respectively, G() is the generator network, y(i,j) represents each pixel unit on the optical image, G(X)(i,j) is each pixel unit on the generated image generated by the generator based on the SAR image, and G(X) represents the image generated by the generator based on X.

[0018] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, the perceptual loss function based on image depth perception distance includes:

[0019]

[0020] Among them, X and Y represent the SAR image and optical image input to the network respectively, h and w represent the length and width of the image matrix respectively, φi represents the feature vector output by the last convolutional layer before the pooling layer, T represents the total number of layers of the feature extraction network, and H i is the length of the image matrix in the i-th layer network, W i is the width of the image matrix in the i-th layer network, w i is the weight factor assigned to the i-th layer, and i represents different network layers.

[0021] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, wherein: the discrete cosine loss function includes:

[0022]

[0023] Where DFT(u,v) is the result of discrete cosine transform, f(x,y) is the input image matrix, N is the width of the image matrix, u and v represent the value of each unit after discrete cosine transform, and x and y represent the value of each unit before transform.

[0024] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection according to the present invention, the discrete cosine loss function further includes:

[0025]

[0026] L DCT (X,Y)=‖DCT(G(X))-DCT(Y)‖ F

[0027] Where DCT() is the discrete cosine transform function, ||·|| Fis the matrix norm, G() is the generator network, X is the input SAR image, Y is the corresponding optical image, and i and j are the scales in the two-dimensional coordinates.

[0028] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, wherein: the image generation network includes,

[0029] Adaptive convolution kernel size:

[0030] C=φ(k)=2 γ*k-b

[0031]

[0032] Where C is the number of channels, |x| odd means to select the nearest odd number, γ=2, b=1, k is the size of the adaptive convolution kernel, is a mapping function.

[0033] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, the loss constraint function includes:

[0034] L GAN (X,Y)=E (X,Y) [logD(X,Y)]+E (X) [log(1-D(X,G(X)))]

[0035] Among them, X and Y represent the SAR image and optical image input to the network respectively, D is the discriminator, and D(X,Y) represents the discriminator's judgment result on X and Y.

[0036] As a preferred solution of the SAR-optical image translation method based on image evaluation and feature selection described in the present invention, wherein: the residual mapping network includes,

[0037] Channel feature selection:

[0038] A c (x)=σ(F(AvgPool(x))+F(MaxPool(x)))

[0039] Among them, A c (x) represents channel feature selection, x represents the input tensor, F(·) represents the convolution operation, AvgPool(·) and MaxPool(·) represent uniform pooling and maximum pooling respectively, and σ(·) represents the Sigmoid activation function;

[0040] Spatial feature selection:

[0041] As (x)=σ(F([AvgPool(x);MaxPool(x)]))

[0042] Among them, A s (x) Spatial feature selection, where [·;·] represents the concatenation operation along the channel dimension.

[0043] The SAR-optical image translation system based on image evaluation and feature selection is characterized by comprising a residual mapping network establishment module, an image generation network establishment module, an algorithm design module and a function implementation module.

[0044] A residual mapping network establishment module, wherein the residual mapping network establishment module is used to establish a residual mapping network based on geometric distortion and low-resolution characteristics of the SAR image in combination with feature selection;

[0045] An image generation network establishment module, wherein the image generation network establishment module is used to establish an image generation network by combining local information, global information and information fusion characteristics;

[0046] A loss function design module, wherein the loss function design module is used to design a loss constraint function based on SAR image generation adversarial, image denoising, and evaluation guidance characteristics;

[0047] A function implementation module is used to design a multi-scale decision network based on the loss constraint algorithm, combine the residual mapping network with the image generation network, and establish an optical image translation model.

[0048] Beneficial effects of the present invention: This invention proposes a SAR-optical image translation method and system based on image evaluation and feature selection. Compared with classic image translation algorithm models such as Pix2pix, CycleGAN, and Pix2pixHD, in complex multi-scene experiments, this model algorithm can better extract effective features from SAR images for mapping, demonstrating the model's superiority in processing complex data sets.

[0049] Compared with classic image translation algorithms such as Pix2pix, CycleGAN, and Pix2pixHD, this model algorithm achieves superior image translation results in four single-scene experiments: farmland, forest, canyon, and river. It also achieves higher scores on four objective image evaluation metrics: SSIM, PSNR, LPIPS, and MSE.

[0050] Classification agent experiments have proved that the proposed algorithm pays more attention to the important information in SAR images that has a strong correspondence with optical images during the feature extraction process, and can better achieve consistency in information expression between optical sensors and SAR sensors.

[0051] Compared with the existing technology, the SAR-optical image translation method based on image evaluation and feature selection proposed in the present invention can achieve better results than the existing image mapping algorithms, whether in complex multi-scene experiments or single-scene experiments of different categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0053] Figure 1 A flowchart of a SAR-optical image translation method and system based on image evaluation and feature selection provided by one embodiment of the present invention;

[0054] Figure 2 A schematic diagram illustrating the operation of a feature selection module of a SAR-optical image translation method and system based on image evaluation and feature selection provided by one embodiment of the present invention;

[0055] Figure 3 A schematic diagram illustrating the operation of an information fusion module of a SAR-optical image translation method and system based on image evaluation and feature selection according to an embodiment of the present invention;

[0056] Figure 4 A specific structural diagram of a residual map generator of a SAR-optical image translation method and system based on image evaluation and feature selection provided by one embodiment of the present invention;

[0057] Figure 5 A specific structural diagram of a multi-scale discriminator of a SAR-optical image translation method and system based on image evaluation and feature selection provided in one embodiment of the present invention;

[0058] Figure 6 A convergence curve diagram of a classification agent experiment of a SAR-optical image translation method and system based on image evaluation and feature selection provided by one embodiment of the present invention;

[0059] Figure 7 This is a comparison of image translation experimental results of a SAR-optical image translation method and system based on image evaluation and feature selection provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0060] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0062] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0063] The present invention is described in detail with reference to schematic diagrams. For ease of illustration, cross-sectional views of device structures may be partially enlarged and not to scale when describing embodiments of the present invention. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0064] In the description of the present invention, it should be noted that the terms "upper, lower, inner, and outer" and other references to orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first, second, or third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0065] In this disclosure, unless otherwise specified or limited, the terms "mounted," "connected," and "connected" should be interpreted broadly. For example, they may refer to fixed, removable, or integral connections. They may also refer to mechanical, electrical, or direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure.

[0066] Example 1

[0067] Reference Figure 1-6, which is the first embodiment of the present invention, provides a SAR-optical image translation method and system based on image evaluation and feature selection, including:

[0068] Step 102: establishing a residual mapping network based on the geometric distortion and low-resolution characteristics of the SAR image in combination with feature selection;

[0069] Specifically, we focus on the more meaningful channel information of the image and use this to filter useful features. By modeling the importance of each channel in the task, we can enhance or suppress different channels, thereby focusing the network's learning attention on the information of interest and preventing the network from mistakenly selecting unimportant and misleading information from too many feature channels.

[0070] Specifically, after the image is decoded, a small-sized high-dimensional feature map is obtained and input into the feature selection module, such as Figure 2 Channel selection compresses the feature tensor from C×H×W to C×1×1 in spatial dimensions through max pooling and average pooling, respectively. Average pooling and max pooling can be used to aggregate the spatial information of feature maps. The resulting compressed tensor is input into a shared weight neural network and then merged element by element. Finally, the corresponding channel selection weight matrix is ​​output through a sigmoid function.

[0071] Furthermore, the residual mapping network includes,

[0072] Channel feature selection:

[0073] A c (x)=σ(F(AvgPool(x))+F(MaxPool(x)))

[0074] Among them, A c (x) represents channel feature selection, x represents the input tensor, F(·) represents the convolution operation, AvgPool(·) and MaxPool(·) represent uniform pooling and maximum pooling respectively, and σ(·) represents the Sigmoid activation function;

[0075] It should be noted that we also need to pay attention to the areas in the image that contribute more to the task.

[0076] Furthermore, the feature tensor is channel-compressed by performing maximum pooling and average pooling operations in the channel dimension respectively. The C×H×W feature tensor is compressed into 1×H×W and then input into a convolutional layer with shared weights. The sum is then added to obtain a dual-channel spatial selection weight matrix.

[0077] Furthermore, by appending this weight matrix to the feature map filtered by channel selection, we can further select feature areas that are more important to the task and discard areas that are not of interest. The specific formula is as follows:

[0078] Spatial feature selection:

[0079] A s (x)=σ(F([AvgPool(x);MaxPool(x)]))

[0080] Among them, A s (x) Spatial feature selection, where [·;·] represents the concatenation operation along the channel dimension.

[0081] Step 104: establishing an image generation network based on the residual mapping network, combining local information, global information, and information fusion characteristics;

[0082] Furthermore, in order to avoid excessive concentration of each selection in the network during feature selection, which would cause local information cocoons and thus affect the diversity and richness of image details, the feature tensor is introduced into the information fusion module during upsampling to carry out appropriate cross-channel interaction.

[0083] It should be noted that the calculation formula for the interaction domain of the information fusion module is as follows:

[0084] The image generation network includes an adaptive convolution kernel size:

[0085] C=φ(k)=2 γ*k-b

[0086]

[0087] Where C is the number of channels, |x| odd means to select the nearest odd number, γ=2, b=1, k is the size of the adaptive convolution kernel, is a mapping function.

[0088] Step 106, designing a loss constraint algorithm based on SAR image generative adversarial, image denoising, and evaluation guidance characteristics;

[0089] It should be noted that the loss constraint algorithm includes:

[0090] L GAN (X,Y)=E (X,Y) [logD(X,Y)]+E (X) [log(1-D(X,G(X)))]

[0091] Among them, X and Y represent the SAR image and optical image input to the network respectively, D is the discriminator, and D(X,Y) represents the discriminator's judgment result on X and Y.

[0092] Furthermore, the error loss function based on mean square error statistics includes:

[0093]

[0094] Among them, X and Y represent the SAR image and optical image input to the network respectively, h and w represent the length and width of the image matrix respectively, i and j represent each pixel unit in the horizontal and vertical directions respectively, G() is the generator network, y(i,j) represents each pixel unit on the optical image, G(X)(i,j) is each pixel unit on the generated image generated by the generator based on the SAR image, and G(X) represents the image generated by the generator based on X.

[0095] Furthermore, the perceptual loss function based on image depth perception distance includes:

[0096]

[0097] Among them, X and Y represent the SAR image and optical image input to the network respectively, h and w represent the length and width of the image matrix respectively, φi represents the feature vector output by the last convolutional layer before the pooling layer, T represents the total number of layers of the feature extraction network, and H i is the length of the image matrix in the i-th layer network, W i is the width of the image matrix in the i-th layer network, w i is the weight factor assigned to the i-th layer, and i represents different network layers.

[0098] Furthermore, the discrete cosine loss function includes,

[0099]

[0100] Where DFT(u,v) is the result of discrete cosine transform, f(x,y) is the input image matrix, N is the width of the image matrix, u and v represent the value of each unit after discrete cosine transform, and x and y represent the value of each unit before transform.

[0101]

[0102] L DCT (X,Y)=‖DCT(G(X))-DCT(Y)‖ F

[0103] Where DCT() is the discrete cosine transform function, ||·|| Fis the matrix norm, G() is the generator network, X is the input SAR image, Y is the corresponding optical image, and i and j are the scales in the two-dimensional coordinates.

[0104] Step 108: Design a multi-scale decision network based on the loss constraint algorithm to implement optical image translation.

[0105] It should be noted that the multi-scale decision network includes:

[0106]

[0107] Among them, λ DCT Represents the weight value of the discrete cosine loss function, λ LPIPS represents the weight value of the perceptual distance loss function based on deep texture structure, λ MSE Represents the weight value of the mean square error loss function based on pixel calibration, L cGAN (G,D k ) represents the loss function based on generative adversarial, L DCT (G,D k ) represents the loss function based on image denoising, L MSE (G,D k ) is the mean square error loss function based on pixel calibration, L LPIPS (G,D k ) is the perceptual distance loss function based on deep texture structure, G * represents the generation network, k is 1-2, representing different discriminant networks, D * Represents the discriminant network, D1 is the first discriminant network, and D2 is the second discriminant network.

[0108] It should be noted that the overall feature selection residual map image generator structure is as follows Figure 4 As shown in the figure, the image encoding module extracts the semantic information of the input image through the downsampling operation of five convolutional layers, and then inputs the extracted feature tensor into the attention residual block for mapping; the resulting mapping output is still an optical image feature tensor of size 16×16×1024, and finally, through five upsampling operations of the inverted convolutional layer, the output optical image of the same size as the input image is obtained.

[0109] Furthermore, after the last two upsamplings, the feature information of the generated image is supplemented and enriched through cross-channel interaction to achieve information fusion. In the network model constructed by the present invention, a multi-scale discriminator is also introduced. The specific structure diagram is as follows Figure 5 shown.

[0110] The SAR-optical image translation system based on image evaluation and feature selection is characterized by comprising a residual mapping network establishment module, an image generation network establishment module, an algorithm design module and a function implementation module.

[0111] A residual mapping network establishment module, wherein the residual mapping network establishment module is used to establish a residual mapping network based on geometric distortion and low-resolution characteristics of the SAR image in combination with feature selection;

[0112] An image generation network establishment module, wherein the image generation network establishment module is used to establish an image generation network by combining local information, global information and information fusion characteristics;

[0113] A loss function design module, wherein the loss function design module is used to design a loss constraint function based on SAR image generation adversarial, image denoising, and evaluation guidance characteristics;

[0114] A function implementation module is used to design a multi-scale decision network based on the loss constraint algorithm, combine the residual mapping network with the image generation network, and establish an optical image translation model.

[0115] The above-mentioned unit modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0116] Example 2

[0117] Reference Figure 1-7 , which is an embodiment of the present invention, provides a SAR-optical image translation method and system based on image evaluation and feature selection. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through comparative experiments.

[0118] To verify that the proposed feature selection residual module can better perform feature screening, we used a classification network, ResNet18, with the same structure as the feature selection residual block, for scene classification proxy experiments. The experimental network incorporated the proposed feature selection module and was named ResNet18AT. Image classification experiments were conducted on both SAR and optical image datasets, with the following network training parameters: a batch size of 16, a training cycle of 100, an SGD optimizer with an optimizer parameter of 0.9 and a learning rate of 0.01.

[0119] Figure 6 This is a convergence curve diagram of the scene classification agent experiment, which proves that the present invention has greatly improved the classification experiment results, the stability of the training network, and the speed of the reasoning process.

[0120] Table 1 Single scene experimental results

[0121]

[0122] From the analysis in Table 1, it can be seen that the straight line features in SAR images are easier to be extracted and mapped by the network, while relatively complex multiple comprehensive features are more difficult to extract. The remote sensing images of farmland scenes have clear farmland structures and clear semantic representations, and they score the highest in the translation task. The pictures in the canyon scenes have inconsistent content. Due to the different perspectives of the pictures, the different canyon shapes, and the vegetation and rocks along the coast, the translation task is more challenging. In forest scenes, our model can better extract and restore texture details in SAR images, thanks to the deep perception structure of our uniformity-constrained images. At the same time, edge features such as river contours and river channels in river scenes can also be well captured by us, which shows that our model can better achieve the extraction of deep and shallow features.

[0123] In order to verify that the image translation algorithm proposed in this invention can achieve better image translation results, this paper conducted translation experiments on 1750 pairs of images from different scenes. The experimental results are as follows: Figure 7 As shown in the figure, from left to right, the first column is the input SAR image, the second to fifth columns are the images generated by Pix2pix, CycleGAN, Pix2pixHD, and the algorithm of our invention, respectively, and the sixth column is the real optical image. The experimental results show that the image translation algorithm proposed in this invention achieves optimal image translation results in different scenarios compared to traditional image translation models.

[0124] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0125] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0130] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A SAR-optical image translation method based on image evaluation and feature selection, characterized by: include, According to the geometric distortion and low-resolution characteristics of SAR images, a residual mapping network is established in combination with feature selection; Combining local information, global information and information fusion characteristics, an image generation network is established; Design loss constraint functions based on SAR image generation adversarial, image denoising, and evaluation guidance features; According to the loss constraint algorithm, a multi-scale decision network is designed, and the residual mapping network is combined with the image generation network to establish an optical image translation model; The multi-scale based decision network includes: Among them, λ DCT Represents the weight value of the discrete cosine loss function, λ LPIPS represents the weight value of the perceptual distance loss function based on deep texture structure, λ MSE Represents the weight value of the mean square error loss function based on pixel calibration, L cGAN (G,D k ) represents the loss function based on generative adversarial, L DCT (G,D k ) represents the loss function based on image denoising, L MSE (G,D k ) is the mean square error loss function based on pixel calibration, L LPIPS (G,D k ) is the perceptual distance loss function based on deep texture structure, G * represents the generative network, D * Denotes the discriminant network, k is 1-2, indicating different discriminant networks, D1 is the first discriminant network, and D2 is the second discriminant network; The error loss function based on mean square error statistics includes: Where X and Y represent the SAR image and optical image input to the network, h and w represent the length and width of the image matrix, i and j represent each pixel unit in the horizontal and vertical directions, G() is the generator network, y(i,j) represents each pixel unit on the optical image, G(X)(i,j) is each pixel unit on the generated image generated by the generator based on the SAR image, and G(X) represents the image generated by the generator based on X; The perceptual loss function based on image depth perception distance includes: Among them, X and Y represent the SAR image and optical image input to the network respectively, h and w represent the length and width of the image matrix respectively, φi represents the feature vector output by the last convolutional layer before the pooling layer, T represents the total number of layers of the feature extraction network, and H i is the length of the image matrix in the i-th layer network, W i is the width of the image matrix in the i-th layer network, w i is the weight factor assigned to the i-th layer, where i represents different network layers; The discrete cosine loss function includes: Where DFT(u,v) is the result of discrete cosine transform, f(x,y) is the input image matrix, N is the width of the image matrix, u,v represent the value of each unit after discrete cosine transform, and x,y represent the value of each unit before transform; The discrete cosine loss function also includes, L DCT (X,Y)=‖DCT(G(X))-DCT(Y)‖ F Where DCT() is the discrete cosine transform function, ||·|| F is the matrix norm, G() is the generator network, X is the input SAR image, Y is the corresponding optical image, and i and j are the scales in the two-dimensional coordinates.

2. The SAR-optical image translation method based on image evaluation and feature selection according to claim 1, wherein: The image generation network includes: Adaptive convolution kernel size: C=φ(k)=2 γ* k-b Where C is the number of channels, |x| odd means to select the nearest odd number, γ=2, b=1, k is the size of the adaptive convolution kernel, is a mapping function.

3. The SAR-optical image translation method based on image evaluation and feature selection according to claim 2, characterized in that: The loss constraint function includes: L GAN (X,Y)=E (X,Y) [logD(X,Y)]+E (X) [log(1-D(X,G(X)))] Among them, X and Y represent the SAR image and optical image input to the network respectively, D is the discriminator, and D(X,Y) represents the discriminator's judgment result on X and Y.

4. The SAR-optical image translation method based on image evaluation and feature selection according to claim 3, wherein: The residual mapping network includes: Channel feature selection: A c (x)=σ(F(AvgPool(x))+F(MaxPool(x))) Among them, A c (x) represents channel feature selection, x represents the input tensor, F(·) represents the convolution operation, AvgPool(·) and MaxPool(·) represent uniform pooling and maximum pooling respectively, and σ(·) represents the Sigmoid activation function; Spatial feature selection: A s (x)=σ(F([AvgPool(x);MaxPool(x)])) Among them, A s (x) Spatial feature selection, where [·;·] represents the concatenation operation along the channel dimension.

5. A SAR-optical image translation system based on image evaluation and feature selection, applying the SAR-optical image translation method based on image evaluation and feature selection according to any one of claims 1 to 4, characterized in that: Including residual mapping network establishment module, image generation network establishment module, algorithm design module and function implementation module, A residual mapping network establishment module, wherein the residual mapping network establishment module is used to establish a residual mapping network based on geometric distortion and low-resolution characteristics of the SAR image in combination with feature selection; An image generation network establishment module, wherein the image generation network establishment module is used to establish an image generation network by combining local information, global information and information fusion characteristics; A loss function design module, wherein the loss function design module is used to design a loss constraint function based on SAR image generation adversarial, image denoising, and evaluation guidance characteristics; A function implementation module is used to design a multi-scale decision network based on the loss constraint algorithm, combine the residual mapping network with the image generation network, and establish an optical image translation model.

Images