Neural network model training method and device, microwave image anti-interference super-resolution reconstruction method and device, electronic equipment and storage medium

Abstract

The invention relates to a neural network model training method and device, an anti-interference super-resolution reconstruction method and device for a microwave image, electronic equipment and a storage medium, and the method comprises the steps: constructing a second optical gray level image set and a simulation microwave image set based on a first optical gray level image set; training a first neural network according to the first optical grayscale image set and the second optical grayscale image set so as to carry out super-resolution processing on any optical grayscale image; an anti-interference super-resolution network is determined through distillation training according to the first optical gray level image set, the simulation microwave image set and the first neural network so as to be used for conducting anti-interference super-resolution reconstruction on any microwave image, and the anti-interference super-resolution network and the first neural network are the same in network structure. According to the embodiment of the invention, a knowledge distillation mechanism can be utilized, so that the anti-interference super-resolution network can realize synchronous image super-resolution reconstruction and interference noise filtering, and the super-resolution reconstruction efficiency is relatively high.

Description

Technical Field

[0001] The present disclosure relates to the technical field of microwave image processing, and in particular, to a method and apparatus for training a neural network model, anti-interference super-resolution reconstruction of microwave images, an electronic device, and a storage medium. Background Art

[0002] During the imaging process of microwave images, interference usually occurs, resulting in interference noise in the obtained microwave images. Affected by this, when performing super-resolution reconstruction on microwave images, the interference noise will also be amplified, making the reconstructed images more severely affected by noise. Therefore, it is very important to suppress interference noise during the super-resolution reconstruction process. Summary of the Invention

[0003] In view of this, the present disclosure provides a technical solution for training a neural network model, a method and apparatus for anti-interference super-resolution reconstruction of microwave images, an electronic device, and a storage medium.

[0004] According to one aspect of the present disclosure, a method for training a neural network model is provided, including: respectively constructing a second optical gray-scale image set and a simulated microwave image set based on a first optical gray-scale image set, wherein any simulated microwave image in the simulated microwave image set is simulated based on a second optical gray-scale image in the second optical gray-scale image set, and any second optical gray-scale image in the second optical gray-scale image set is downsampled from a first optical gray-scale image in the first optical gray-scale image set, and the resolution of the second optical gray-scale image is lower than that of its corresponding first optical gray-scale image; training a first neural network according to the first optical gray-scale image set and the second optical gray-scale image set, wherein the first neural network is used for performing super-resolution processing on any optical gray-scale image; determining an anti-interference super-resolution network through distillation training according to the first optical gray-scale image set, the simulated microwave image set, and the first neural network, wherein the anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used for performing anti-interference super-resolution reconstruction on any microwave image.

[0005] In a possible implementation, constructing a second optical grayscale image set and a simulated microwave image set based on the first optical grayscale image set respectively includes: for any first optical grayscale image in the first optical grayscale image set, performing downsampling processing on the first optical grayscale image to determine the second optical grayscale image corresponding to the first optical grayscale image; determining the second optical grayscale image set according to the second optical grayscale images corresponding to each first optical grayscale image in the first optical grayscale image set; for any second optical grayscale image in the second optical grayscale image set, performing speckle noise simulation on the second optical grayscale image to determine the simulated microwave image corresponding to the second optical grayscale image; determining the simulated microwave image set according to the simulated microwave images corresponding to each second optical grayscale image in the second optical grayscale image set.

[0006] In a possible implementation, the step of, for any second optical grayscale image in the second optical grayscale image set, performing speckle noise simulation on the second optical grayscale image to determine the simulated microwave image corresponding to the second optical grayscale image includes: based on the gamma distribution, determining a multiplicative noise simulation function, where the multiplicative noise simulation function is used to simulate speckle noise; for any second optical grayscale image in the second optical grayscale image set, performing speckle noise simulation on the second optical grayscale image according to the multiplicative noise simulation function to determine the simulated microwave image corresponding to the second optical grayscale image.

[0007] In a possible implementation, the step of determining the anti-interference super-resolution network through distillation training according to the first optical grayscale image set, the simulated microwave image set, and the first neural network includes: inputting any second optical grayscale image in the second optical grayscale image set into the first neural network to respectively determine the first encoded feature corresponding to the first neural network and the super-resolution reconstruction image of the second optical grayscale image; inputting the simulated microwave image corresponding to the second optical grayscale image into the anti-interference super-resolution network to determine the second encoded feature corresponding to the anti-interference super-resolution network and the super-resolution reconstruction image of the simulated microwave image; iteratively training the anti-interference super-resolution network according to the first encoded feature, the second encoded feature, the super-resolution reconstruction image of the second optical grayscale image, the super-resolution reconstruction image of the simulated microwave image, and a preset multi-objective loss function until a preset training condition is reached, and determining the trained anti-interference super-resolution network.

[0008] In a possible implementation, the multi-objective loss function includes at least one of the following: reconstruction loss, feature distillation loss, target distillation loss, and gradient loss; wherein, the reconstruction loss is used to quantify the difference between the super-resolution reconstruction image obtained by passing any simulated microwave image through the anti-interference super-resolution network and the first optical grayscale image corresponding to the simulated microwave image; the feature distillation loss is used to quantify the feature distance between the first encoded feature and the second encoded feature; the target distillation loss is used to quantify the difference between the super-resolution reconstruction image obtained by passing a simulated microwave image through the anti-interference super-resolution network and the super-resolution reconstruction image obtained by passing the second optical grayscale image corresponding to the simulated microwave image through the first neural network; the gradient loss is used to retain the texture details of the super-resolution reconstruction image output by the anti-interference super-resolution network.

[0009] According to another aspect of the present disclosure, there is provided a method for anti-interference super-resolution reconstruction of microwave images, including: determining a microwave image to be reconstructed; inputting the microwave image to be reconstructed into an anti-interference super-resolution network to determine a super-resolution reconstruction image corresponding to the microwave image to be reconstructed, wherein the anti-interference super-resolution network is trained by the above method, and the resolution of the super-resolution reconstruction image is higher than the resolution of the microwave image to be reconstructed.

[0010] According to another aspect of the present disclosure, there is provided a neural network model training device, including: a microwave image simulation module, configured to respectively construct a second optical grayscale image set and a simulated microwave image set based on a first optical grayscale image set, wherein any simulated microwave image in the simulated microwave image set is simulated based on a second optical grayscale image in the second optical grayscale image set, and any second optical grayscale image in the second optical grayscale image set is downsampled from a first optical grayscale image in the first optical grayscale image set, and the resolution of the second optical grayscale image is lower than the resolution of its corresponding first optical grayscale image; a first network training module, configured to train a first neural network according to the first optical grayscale image set and the second optical grayscale image set, wherein the first neural network is used to perform super-resolution processing on any optical grayscale image; a second network training module, configured to determine an anti-interference super-resolution network through distillation training according to the first optical grayscale image set, the simulated microwave image set, and the first neural network, wherein the anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used to perform anti-interference super-resolution reconstruction on any microwave image.

[0011] According to another aspect of the present disclosure, there is provided an anti-interference super-resolution reconstruction device for microwave images, including: an image acquisition module configured to determine a microwave image to be reconstructed; a super-resolution reconstruction module configured to input the microwave image to be reconstructed into an anti-interference super-resolution network to determine a super-resolution reconstruction image corresponding to the microwave image to be reconstructed, wherein the anti-interference super-resolution network is trained by the above method, and the resolution of the super-resolution reconstruction image is higher than that of the microwave image to be reconstructed.

[0012] According to another aspect of the present disclosure, there is provided an electronic device including a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the above method.

[0013] According to another aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the above method.

[0014] In the embodiments of the present disclosure, in view of the problem that there is no interference-free microwave image in actual applications and it is difficult to obtain paired high-resolution microwave images and low-resolution microwave images, a second optical gray-scale image set and a simulated microwave image set can be respectively constructed based on the first optical gray-scale image set. Any simulated microwave image in the simulated microwave image set is simulated based on a second optical gray-scale image in the second optical gray-scale image set. Any second optical gray-scale image in the second optical gray-scale image set is downsampled from a first optical gray-scale image in the first optical gray-scale image set, and the resolution of the second optical gray-scale image is lower than that of its corresponding first optical gray-scale image. Thus, the first optical gray-scale image can be used as a high-resolution and interference-free microwave image, and the second optical gray-scale image can be used as a low-resolution and interference-free microwave image, cooperating with the simulated microwave image with interference noise, to provide feasibility for the supervised training model based on the neural network. According to the first optical gray-scale image set and the second optical gray-scale image set, a first neural network for performing super-resolution processing on any optical gray-scale image can be trained to serve as a teacher network for guiding the subsequent anti-interference super-resolution network; according to the first optical gray-scale image set, the simulated microwave image set, and the first neural network, through distillation training, the influence of interference noise in the super-resolution image reconstruction process of the simulated microwave image is eliminated by using the first neural network, and an anti-interference super-resolution network having the same network structure as the first neural network is determined, so as to realize synchronous filtering of interference noise and super-resolution reconstruction in the microwave image, such that the trained anti-interference super-resolution network can perform anti-interference super-resolution reconstruction on any microwave image, improving the efficiency of super-resolution reconstruction and the image quality of the reconstructed image.

[0015] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are included in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0017] Figure 1 A flowchart showing a method for training a neural network model according to an embodiment of the present disclosure;

[0018] Figure 2 A schematic diagram showing the principle of distillation training according to an embodiment of the present disclosure;

[0019] Figure 3 A schematic diagram showing the process of feature distillation according to an embodiment of the present disclosure;

[0020] Figure 4 A flowchart showing a method for anti-interference super-resolution reconstruction of microwave images according to an embodiment of the present disclosure;

[0021] Figure 5 A block diagram showing an apparatus for training a neural network model according to an embodiment of the present disclosure;

[0022] Figure 6 A block diagram showing an apparatus for anti-interference super-resolution reconstruction of microwave images according to an embodiment of the present disclosure;

[0023] Figure 7 A block diagram showing an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote like or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0025] As used herein, the terms "include," "comprise," "have," or variations thereof are open-ended and include one or more stated features, wholes, elements, steps, components, or functions, but do not exclude the existence or addition of one or more other features, wholes, elements, steps, components, functions, or groups thereof.

[0026] When an element is referred to as being "connected," "coupled," "responsive," or variations thereof to another element, it can be directly connected, coupled, or responsive to the other element, or intervening elements may be present.

[0027] Although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Thus, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0028] As used herein, the term "exemplary" means "serving as an example, instance, or illustration." Any embodiment so described herein is not necessarily to be construed as preferred or better than other embodiments.

[0029] As used herein, the term "and / or" is merely a description of an associative relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, as used herein, the term "at least one" means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0030] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0031] Microwave imaging technology can penetrate clouds, rain, and vegetation, providing information that cannot be obtained under visible light and infrared conditions, and has unique advantages in application fields such as weather forecasting, remote sensing, and radar imaging. However, due to factors such as the relatively long wavelength of microwaves compared to the wavelengths of visible light and infrared, the size limitation of antennas in microwave imaging systems, the susceptibility of microwave transmission to atmospheric conditions such as rainfall and humidity, and the system complexity of microwave imaging systems, the resolution of microwave images is usually relatively low. Considering aspects such as manufacturing cost and operation difficulty, in the prior art, super-resolution processing is usually performed on microwave images to improve the resolution of microwave images.

[0032] However, due to the coherent superposition of many basic echoes within the radar resolution unit, microwave images are often interfered by speckle noise. In addition, microwave signals are affected by various environmental factors during transmission, such as water vapor, raindrops, snowflakes, etc. in the atmosphere. These factors will scatter and absorb microwave signals, resulting in signal attenuation and distortion, and will also cause interference noise. Therefore, when performing super-resolution reconstruction on microwave images, the interference noise will also be amplified, making the reconstructed images more severely affected by noise. Therefore, it is very important to suppress interference noise during the super-resolution reconstruction process.

[0033] In the prior art, the joint anti-interference super-resolution reconstruction of microwave images can be achieved by cascading common super-resolution and anti-interference algorithms in image processing. However, this method will introduce additional time overhead, resulting in low efficiency of image super-resolution reconstruction, and thus limiting the application of microwave images, especially in mobile microwave imaging devices. On the other hand, this method is easily affected by error accumulation and information loss in each model. For low-resolution microwave images with interference, if super-resolution is performed on them first, the interference will be amplified, making subsequent interference suppression more difficult; if interference suppression is performed on them first, it may over-smooth the image details, resulting in a more blurred subsequent super-resolution result.

[0034] In view of this, the present disclosure provides a neural network model training method, which can combine the image super-resolution reconstruction and anti-interference functions by using the knowledge distillation mechanism, so that the anti-interference super-resolution network can synchronously perform image super-resolution reconstruction and interference noise filtering, and has a high super-resolution reconstruction efficiency. The neural network model training method provided by the present disclosure is introduced in detail below.

[0035] Figure 1 The flowchart of a neural network model training method according to an embodiment of the present disclosure is shown. This neural network model training method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. This neural network model training method can be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the neural network model training method can be executed by a server. As Figure 1 shown, this neural network model training method includes:

[0036] In step S11, based on the first set of optical grayscale images, a second set of optical grayscale images and a set of simulated microwave images are respectively constructed. Any one of the simulated microwave images in the set of simulated microwave images is simulated based on one of the second optical grayscale images in the second set of optical grayscale images. Any one of the second optical grayscale images in the second set of optical grayscale images is downsampled from one of the first optical grayscale images in the first set of optical grayscale images, and the resolution of the second optical grayscale image is lower than that of its corresponding first optical grayscale image.

[0037] To train a neural network model capable of performing image super-resolution reconstruction tasks, paired low-resolution images and high-resolution images are required. Therefore, it is very difficult to use the same microwave imaging device to capture at least two microwave images that are exactly the same except for the resolution for the same target object in actual microwave imaging technology applications. Especially in the field of remote sensing microwave imaging for regions, not only the difference in imaging resolution needs to be considered, but also the time interval between different images cannot be too long to avoid changes in the ground environment. Therefore, it is usually difficult to directly obtain sample microwave images for neural network training.

[0038] To address this issue, in the embodiments of the present disclosure, the second set of optical grayscale images and the set of simulated microwave images can be constructed by downsampling the first optical grayscale images with relatively high resolution in the first set of optical grayscale images to obtain second optical grayscale images with lower resolution, and then performing simulation processing based on the second optical grayscale images to obtain corresponding simulated microwave images. On this basis, each first optical grayscale image can be regarded as a high-resolution microwave image without interference noise and paired with the simulated microwave image as a paired training sample, thereby providing a data basis for the subsequent supervised training of the neural network model and making the supervised training of the anti-interference super-resolution network feasible.

[0039] The first set of optical grayscale images here includes multiple first optical grayscale images without interference noise. The specific number of the first optical grayscale images can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations in this regard. The specific method for determining any one of the first optical grayscale images can refer to the implementation methods in related technologies, and the present disclosure does not make specific limitations in this regard.

[0040] Among them, the specific method for downsampling the first optical grayscale image can refer to the implementation methods in related technologies. For example, bilinear interpolation method or regional average downsampling method can be used, and the present disclosure does not make specific limitations in this regard.

[0041] The specific method for simulating the second optical grayscale image can be flexibly set according to actual usage requirements. For example, fixed interference noise can be directly added to the second optical grayscale image, etc. The present disclosure does not make specific limitations in this regard.

[0042] Subsequently, in combination with possible implementation manners of the present disclosure, the process of respectively constructing the second optical grayscale image set and the simulated microwave image set based on the first optical grayscale image set will be described in detail, and will not be elaborated here.

[0043] In step S12, a first neural network is trained according to the first optical grayscale image set and the second optical grayscale image set, where the first neural network is used to perform super-resolution processing on any optical grayscale image.

[0044] Generally, for two neural network models with the same encoder-decoder structure, when the intermediate layer latent variables corresponding to these two neural network models are the same, for the same input data, the output data of these two neural network models is also the same. That is to say, for a fixed decoder, the higher the similarity of the intermediate layer latent variables corresponding to these two neural network models, the higher the similarity of the output data of these two neural network models.

[0045] Based on the above principle, it can be known that when using a neural network model to perform super-resolution reconstruction on a microwave image, interference-free latent variables can be introduced during the super-resolution reconstruction process through feature distillation, which can eliminate the influence of interference noise, so as to be able to reconstruct a high-resolution microwave image without interference noise and achieve anti-interference super-resolution reconstruction.

[0046] Specifically, the first neural network preset can be iteratively trained using the first optical grayscale image set and the second optical grayscale image set without interference noise, so that the trained first neural network can complete the super-resolution reconstruction task for the optical grayscale image and provide interference-free latent variables to guide the subsequent training of the anti-interference super-resolution network. Among them, the specific structure of the first neural network can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations in this regard. Preferably, the first neural network should have an encoder-sampling layer-decoder structure to facilitate flexible matching of different modules and different loss functions, and provide flexibility for the neural network model training method of the embodiments of the present disclosure.

[0047] The specific method for training the first neural network can refer to the implementation manners in related technologies, and the present disclosure does not make specific limitations in this regard.

[0048] In a possible implementation, for any first optical grayscale image in the first optical grayscale image set, the first optical grayscale image can be dimensionally reconstructed to obtain a reconstructed image corresponding to the first optical grayscale image; the reconstructed image corresponding to the first optical grayscale image and the second optical grayscale image corresponding to the first optical grayscale image are spliced and input into the first neural network to obtain an initial reconstructed image corresponding to the first optical grayscale image; according to each first optical grayscale image, each initial reconstructed image corresponding to the first optical grayscale image, and a preset reconstruction loss function, the first neural network is iteratively trained until a preset iteration termination condition is satisfied, and a trained first neural network is obtained.

[0049] Specifically, in order to guide the anti-interference super-resolution network to filter interference noise during the super-resolution reconstruction process, the first neural network needs to provide interference-free latent variables. Therefore, the training data used by the first neural network is the first optical grayscale image set and the second optical grayscale image set without interference noise. However, when downsampling the first optical grayscale image, it usually causes the loss of some image information, making the second optical grayscale image unable to fully reflect the distribution of the first optical grayscale image. Therefore, the first optical grayscale image can be dimensionally reconstructed to obtain a reconstructed image corresponding to the first optical grayscale image; then, the reconstructed image corresponding to the first optical grayscale image and the second optical grayscale image are spliced as the input for training the first neural network, so that the first neural network can fully learn the distribution of the first optical grayscale image and improve the performance of the first neural network.

[0050] Among them, the specific method of dimensional reconstruction and the specific method of image splicing can both refer to the implementation methods in related technologies, and the present disclosure does not make specific limitations on this.

[0051] The reconstruction loss function is used to quantify the difference between any first optical grayscale image and the initial reconstructed image corresponding to the first optical grayscale image. Its specific form can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations on this.

[0052] In an example, the reconstruction loss function can be expressed as formula (1):

[0053] L T =||Tea(X LR ,X HR )-X HB || (1)

[0054] Among them, L T represents the reconstruction loss; X HR represents the reconstructed image corresponding to any first optical grayscale image; X LR represents the second optical grayscale image corresponding to the first optical grayscale image; (XLR ,X HR ) represents the splicing result of the second optical grayscale image corresponding to the first optical grayscale image and the reconstructed image corresponding to the first optical grayscale image; Tea(X LR ,X HR ) represents the initial reconstructed image obtained through the first neural network.

[0055] The iteration termination condition here can be flexibly set according to actual usage requirements. For example, it can be set to meet a preset number of iterations, or the overall loss function converges, etc. The present disclosure does not make specific limitations on this.

[0056] In step S13, according to the first optical grayscale image set, the simulated microwave image set, and the first neural network, an anti-interference super-resolution network is determined through distillation training. Among them, the anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used for anti-interference super-resolution reconstruction of any microwave image.

[0057] Figure 2 Shows a schematic diagram of the principle of a distillation training according to an embodiment of the present disclosure. As Figure 2 shown, after the training of the first neural network for completing the super-resolution reconstruction task of the optical grayscale image, the first neural network can be used as a teacher network. During the process of the first neural network performing super-resolution reconstruction on any optical grayscale image, based on feature distillation, the intermediate layer latent variables corresponding to the first neural network are extracted to train the preset anti-interference super-resolution network, so that the trained anti-interference super-resolution network can be used for anti-interference super-resolution network of any microwave image.

[0058] It should be noted that in order to make full use of the intermediate layer latent variables of the first neural network and ensure that the anti-interference super-resolution network can effectively filter interference noise in super-resolution reconstruction, the network structure of the anti-interference super-resolution network should be the same as that of the first neural network; however, the number of channels corresponding to each network layer of the anti-interference super-resolution network can be only half of the number of channels of the corresponding network layer in the first neural network.

[0059] The process of determining the anti-interference compression network through distillation training using the first optical grayscale image set, the simulated microwave image set, and the first neural network will be described in detail later in combination with possible implementation manners of the present disclosure, and will not be elaborated here.

[0060] In the embodiments of the present disclosure, for the problem that there is no interference-free microwave image in practical applications and it is difficult to obtain paired high-resolution microwave images and low-resolution microwave images, a second optical gray-scale image set and a simulated microwave image set can be constructed respectively based on the first optical gray-scale image set. Among them, any simulated microwave image in the simulated microwave image set is simulated based on a second optical gray-scale image in the second optical gray-scale image set. Any second optical gray-scale image in the second optical gray-scale image set is downsampled from a first optical gray-scale image in the first optical gray-scale image set, and the resolution of the second optical gray-scale image is lower than that of its corresponding first optical gray-scale image. Thus, the first optical gray-scale image can be used as a high-resolution and interference-free microwave image, and the second optical gray-scale image can be used as a low-resolution and interference-free microwave image, cooperating with the simulated microwave image with interference noise, to provide feasibility for the supervised training model based on the neural network. According to the first optical gray-scale image set and the second optical gray-scale image set, a first neural network for super-resolution processing of any optical gray-scale image can be trained to serve as the teacher network for guiding the anti-interference super-resolution network in the subsequent stage; according to the first optical gray-scale image set, the simulated microwave image set, and the first neural network, through distillation training, the influence of interference noise in the super-resolution image reconstruction process of the simulated microwave image is eliminated by the first neural network, and an anti-interference super-resolution network with the same network structure as the first neural network is determined, so as to realize the synchronous filtering of interference noise and super-resolution reconstruction in the microwave image, enabling the trained anti-interference super-resolution network to perform anti-interference super-resolution reconstruction on any microwave image and improving the efficiency of super-resolution reconstruction and the image quality of the reconstructed image.

[0061] In a possible implementation manner, constructing a second optical gray-scale image set and a simulated microwave image set respectively based on the first optical gray-scale image set includes: for any first optical gray-scale image in the first optical gray-scale image set, performing downsampling processing on the first optical gray-scale image to determine the second optical gray-scale image corresponding to the first optical gray-scale image; determining the second optical gray-scale image set according to the second optical gray-scale images corresponding to each first optical gray-scale image in the first optical gray-scale image set; for any second optical gray-scale image in the second optical gray-scale image set, performing speckle noise simulation on the second optical gray-scale image to determine the simulated microwave image corresponding to the second optical gray-scale image; determining the simulated microwave image set according to the simulated microwave images corresponding to each second optical gray-scale image in the second optical gray-scale image set.

[0062] Common microwave imaging techniques in the prior art, such as SAR, etc., usually use coherent electromagnetic waves such as microwaves for imaging. When the coherent electromagnetic wave irradiates the imaging target, it will interact with the rough surface of the imaging target or multiple scatterers in the medium, and due to the length difference of the echo paths corresponding to different scatterers, a corresponding phase change will occur, and then a coherence phenomenon will be generated, resulting in granular brightness changes in the microwave image, that is, speckle noise.

[0063] Based on the above principle, for any first optical gray-scale image in the first optical gray-scale image set, the first optical gray-scale image can be first downsampled to determine the second optical gray-scale image corresponding to the first optical gray-scale image; then the second optical gray-scale image is subjected to speckle noise simulation to obtain the simulated microwave image corresponding to the second optical gray-scale image, thereby constructing the matching relationship of the high-resolution first optical gray-scale image - low-resolution second optical gray-scale image - low-resolution simulated microwave image, and determining the second optical gray-scale image set and the simulated microwave image set.

[0064] Among them, the specific method of speckle noise simulation can be flexibly set according to actual usage requirements. For example, the statistical characteristics common to speckle noise can be used to adjust the brightness value corresponding to each pixel value in the optical gray-scale image, etc. The present disclosure does not make specific limitations on this.

[0065] In a possible implementation manner, for any second optical gray-scale image in the second optical gray-scale image set, performing speckle noise simulation on the second optical gray-scale image to determine the simulated microwave image corresponding to the second optical gray-scale image includes: determining a multiplicative noise simulation function based on the gamma distribution, where the multiplicative noise simulation function is used to simulate speckle noise; for any second optical gray-scale image in the second optical gray-scale image set, performing speckle noise simulation on the second optical gray-scale image according to the multiplicative noise simulation function to determine the simulated microwave image corresponding to the second optical gray-scale image.

[0066] Specifically, the speckle noise in the microwave image can usually be described by multiplicative noise. Therefore, a multiplicative noise simulation function can be determined based on the gamma distribution (Gamma Distribution) to be used to simulate speckle noise. Among them, the gamma distribution can be used to describe the intensity statistical characteristics of the microwave image.

[0067] For any second optical gray-scale image in the optical gray-scale image set, the multiplicative noise simulation function can be multiplied by the second optical gray-scale image, thereby performing speckle noise simulation to determine the simulated microwave image corresponding to the second optical gray-scale image. The simulated microwave image obtained according to this simulation method can be expressed by formulas (2) and (3):

[0068] Y(i, j) = X(i, j)N(i, j) (2)

[0069]

[0070] Wherein, X(i, j) represents any second optical grayscale image; Y(i, j) represents the simulated microwave image corresponding to the second optical grayscale image; N(i, j) and N represent multiplicative noise simulation functions; P N (N) represents the probability density function corresponding to the multiplicative noise simulation function; L represents the number of looks, which is used to describe the average number of times in multi-look processing to control the noise intensity; Г(L) represents the gamma function, and the mean value corresponding to this gamma function is 1, and the variance is 1 / L.

[0071] In addition, in some usage scenarios, the Rayleigh Distribution can also be used to determine the multiplicative noise simulation function to perform speckle noise simulation on the second optical grayscale image.

[0072] In a possible implementation manner, according to the first optical grayscale image set, the simulated microwave image set, and the first neural network, an anti-interference super-resolution network is determined through distillation training, including: inputting any second optical grayscale image in the second optical grayscale image set into the first neural network to respectively determine the first encoded feature corresponding to the first neural network and the super-resolution reconstructed image of the second optical grayscale image; inputting the simulated microwave image corresponding to the second optical grayscale image into the anti-interference super-resolution network to determine the second encoded feature corresponding to the anti-interference super-resolution network and the super-resolution reconstructed image of the simulated microwave image; and iteratively training the anti-interference super-resolution network according to the first encoded feature, the second encoded feature, the super-resolution reconstructed image of the second optical grayscale image, the super-resolution reconstructed image of the simulated microwave image, and a preset multi-objective loss function until a preset training condition is reached to determine the trained anti-interference super-resolution network.

[0073] Figure 3 Shows a schematic diagram of the principle of a feature distillation according to an embodiment of the present disclosure. As Figure 3As shown, the first neural network represents any neural network for the task of image super-resolution reconstruction and having an encoder-sampling layer-decoder network structure. After any image is input into the first neural network, the decoder first converts the image into a latent feature space, then extracts image features from coarse to fine through a series of multi-scale residual blocks (MultiResBlock, MRB), and uses an attention module (AttentionModule, AM) to enhance important feature parts to obtain multi-level latent features. At the same time, the output of the shallow layer is fused into subsequent network layers to prevent the loss of edge information and texture information. Among them, the latent features can be expressed by formulas (4) to (6):

[0074] Z1 = E1(Conv(X)) (4)

[0075] Z2 = E2(Z1 + Conv(X)) (5)

[0076] Z3 = E3(Z2 + Conv(X)) (6)

[0077] Among them, X represents the image input into the first neural network; Conv(X) represents the convolution of the image; Z1 represents the first-layer latent feature of the encoder; Z2 represents the second-layer latent feature of the encoder; Z3 represents the second-layer latent feature of the encoder; both E1 and E2 are network layers composed of two MRBs and one AM, and E3 is a network layer composed of two MRBs.

[0078] The upsampling module will upsample the multi-level latent features and fuse them into super-resolution features; the super-resolution features will be input into the decoder to be reconstructed into a super-resolution reconstructed image. Among them, the super-resolution features can be expressed by formula (7):

[0079] Z SR = S(Z1) + S(Z2) + S(Z3) (7)

[0080] Among them, Z SR represents the super-resolution features; S(·) represents the sub-pixel convolution function.

[0081] Based on the above principle, in order to enable the anti-interference super-resolution network to learn interference-free latent variables from the first neural network and make the latent representations of the two as similar as possible, it can be sorted out during the encoding stage of the first neural network, and the first-layer latent feature corresponding to the first neural network is extracted as the latent variable for training the anti-interference super-resolution network.

[0082] Specifically, any second optical grayscale image in the second optical grayscale image set can be input into the first neural network for feature distillation, and at least one layer of latent features corresponding to the first neural network can be extracted as the first encoded feature. Similarly, the simulated microwave image corresponding to the second optical grayscale image is input into the anti-interference super-resolution network for feature distillation, and at least one layer of latent features corresponding to the anti-interference super-resolution network is extracted as the second encoded feature, so as to perform distillation training on the anti-interference super-resolution network by analyzing the difference between the first encoded feature and the second encoded feature.

[0083] On the other hand, in order to enable the anti-interference super-resolution network to learn the data distribution from the first neural network and the first optical grayscale image simultaneously during the training process, target distillation can also be introduced in the distillation training, that is, minimizing the feature distance between the super-resolution reconstructed image output by the anti-interference super-resolution network and the super-resolution reconstructed image output by the first neural network to improve the performance of the anti-interference super-resolution network.

[0084] Based on the first encoded feature, the second encoded feature, the super-resolution reconstructed image of the second optical grayscale image, the super-resolution reconstructed image of the simulated microwave image, and a preset multi-objective loss function, the anti-interference super-resolution network can be iteratively trained until the preset training conditions are met, and the trained anti-interference super-resolution network is determined. The preset training conditions here can be flexibly set according to actual usage requirements. For example, it can be set to meet the preset number of iterations, or the multi-objective loss function converges, etc. The present disclosure does not make specific limitations on this.

[0085] Among them, the specific form of the multi-objective loss function can be flexibly set according to actual usage requirements, and it should be able to introduce the difference between the first encoded feature and the second encoded feature, as well as the feature distance between the super-resolution reconstructed image output by the anti-interference super-resolution network and the super-resolution reconstructed image output by the first neural network. The present disclosure does not make specific limitations on this.

[0086] In a possible implementation, the multi-objective loss function includes at least one of the following: reconstruction loss, feature distillation loss, target distillation loss, and gradient loss; where the reconstruction loss is used to quantify the difference between the super-resolution reconstructed image obtained by the anti-interference super-resolution network for any simulated microwave image and the first optical grayscale image corresponding to the simulated microwave image; the feature distillation loss is used to quantify the feature distance between the first encoded feature and the second encoded feature; the target distillation loss is used to quantify the difference between the super-resolution reconstructed image obtained by the anti-interference super-resolution network for the simulated microwave image and the super-resolution reconstructed image obtained by the first neural network for the second optical grayscale image corresponding to the simulated microwave image; the gradient loss is used to retain the texture details of the super-resolution reconstructed image output by the anti-interference super-resolution network.

[0087] Specifically, the reconstruction loss can be used to quantify the difference between the super-resolution reconstruction image obtained by the anti-interference super-resolution network for any simulated microwave image and the first optical grayscale image corresponding to the simulated microwave image. Its specific form can be flexibly set according to actual usage requirements. For example, reference can be made to formula (1) above, etc. The present disclosure does not make specific limitations thereto.

[0088] The feature distillation loss can be used to quantify the feature distance between the first encoded feature and the second encoded feature. Its specific form can be flexibly set according to actual usage requirements. The present disclosure does not make specific limitations thereto.

[0089] In one example, the feature distillation loss can be expressed as formula (8):

[0090]

[0091] where L FD represents the feature distillation loss; Z l T represents the l-th layer latent feature corresponding to the first neural network; Z l S represents the l-th layer latent feature corresponding to the anti-interference super-resolution network; Z represents the number of elements in the feature layer; N represents the number of layers for which feature distillation is performed.

[0092] The target distillation loss can be used to quantify the difference between the super-resolution reconstruction image obtained by the anti-interference super-resolution network for the simulated microwave image and the super-resolution reconstruction image obtained by the first neural network for the second optical grayscale image corresponding to the simulated microwave image. Its specific form can be flexibly set according to actual usage requirements. The present disclosure does not make specific limitations thereto.

[0093] In one example, for any first optical grayscale image in the first optical grayscale image set, the dimension of the first optical grayscale image can be reconstructed to obtain the reconstructed image corresponding to the first optical grayscale image; the reconstructed image corresponding to the first optical grayscale image and the second optical grayscale image corresponding to the first optical grayscale image are spliced and input into the first neural network for image reconstruction to obtain the corresponding initial reconstruction image; the simulated microwave image corresponding to the first optical grayscale image is input into the anti-interference super-resolution network to obtain the corresponding super-resolution reconstruction image; then the target distillation loss can be expressed as formula (9):

[0094]

[0095] where L TD represents the target distillation loss; x HRdenotes the reconstructed image corresponding to any first optical grayscale image; x LR denotes the second optical grayscale image corresponding to the first optical grayscale image; denotes the simulated microwave image corresponding to the first optical grayscale image; denotes the super-resolution reconstructed image obtained by the anti-interference super-resolution network for the simulated microwave image; (X LR , X HR ) denotes the splicing result of the reconstructed image corresponding to the first optical grayscale image and the second optical grayscale image corresponding to the first optical grayscale image; Tea(X LR , X HR ) denotes the super-resolution reconstructed image obtained by the first neural network.

[0096] The gradient loss is used to retain the texture details of the super-resolution reconstructed image output by the anti-interference super-resolution network; its specific form can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations thereon.

[0097] In one example, the gradient loss can be expressed as formula (10):

[0098]

[0099] wherein, denotes the gradient loss; denotes the gradient in the horizontal direction; denotes the gradient in the vertical direction.

[0100] Based on the above formula (1), formula (8), formula (9) and formula (10), the multi-objective rate-distortion loss function can be expressed as formula (11):

[0101]

[0102] wherein, L S denotes the multi-objective rate-distortion loss function; α1, α2 and α3 denote loss weights, and their specific values can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations thereon.

[0103] In the embodiments of the present disclosure, for the problem that there is no interference-free microwave image in practical applications and it is difficult to obtain paired high-resolution microwave images and low-resolution microwave images, a second optical gray-scale image set and a simulated microwave image set can be respectively constructed based on the first optical gray-scale image set. Among them, any simulated microwave image in the simulated microwave image set is simulated based on a second optical gray-scale image in the second optical gray-scale image set, and any second optical gray-scale image in the second optical gray-scale image set is downsampled from a first optical gray-scale image in the first optical gray-scale image set, and the resolution of the second optical gray-scale image is lower than that of its corresponding first optical gray-scale image. Thus, the first optical gray-scale image can be used as a high-resolution and interference-free microwave image, and the second optical gray-scale image can be used as a low-resolution and interference-free microwave image. Together with the simulated microwave image with interference noise, it provides feasibility for the supervised training model based on the neural network. According to the first optical gray-scale image set and the second optical gray-scale image set, a first neural network for super-resolution processing of any optical gray-scale image can be trained to serve as a teacher network for guiding the anti-interference super-resolution network subsequently; according to the first optical gray-scale image set, the simulated microwave image set, and the first neural network, through distillation training, the influence of interference noise in the super-resolution image reconstruction process of the simulated microwave image is eliminated by the first neural network, and an anti-interference super-resolution network with the same network structure as the first neural network is determined, so as to realize the synchronous filtering of interference noise and super-resolution reconstruction in the microwave image, enabling the trained anti-interference super-resolution network to perform anti-interference super-resolution reconstruction on any microwave image and improving the efficiency of super-resolution reconstruction and the image quality of the reconstructed image.

[0104] The present disclosure also provides an anti-interference super-resolution reconstruction method for microwave images.

[0105] Figure 4 The flowchart of an anti-interference super-resolution reconstruction method for microwave images according to an embodiment of the present disclosure is shown. This anti-interference super-resolution reconstruction method for microwave images can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. This anti-interference super-resolution reconstruction method for microwave images can be implemented by the processor calling computer-readable instructions stored in the memory. Alternatively, the anti-interference super-resolution reconstruction method for microwave images can be executed by the server. As Figure 4 shown, this anti-interference super-resolution reconstruction method for microwave images includes:

[0106] In step S41, a microwave image to be reconstructed is determined.

[0107] In step S42, the microwave image to be reconstructed is input into the anti-interference super-resolution network to determine a super-resolution reconstructed image corresponding to the microwave image to be reconstructed. The anti-interference super-resolution network is trained by the above method, and the resolution of the super-resolution reconstructed image is higher than that of the microwave image to be reconstructed.

[0108] The microwave image to be reconstructed here can represent any microwave image that needs to be super-resolved. The specific form of the microwave image to be reconstructed can be flexibly set according to actual usage requirements. For example, it can be a SAR image, etc. The present disclosure does not make specific limitations on this.

[0109] After the microwave image to be reconstructed is input into the anti-interference super-resolution network, anti-interference super-resolution can be directly performed using the anti-interference super-resolution network to obtain a super-resolution reconstructed image with a resolution higher than that of the microwave image to be reconstructed.

[0110] In the embodiment of the present disclosure, after determining the microwave image to be reconstructed, the microwave image to be reconstructed can be input into the anti-interference super-resolution network trained by the above method to directly and quickly determine the super-resolution reconstructed image corresponding to the microwave image to be reconstructed, combining image super-resolution reconstruction and anti-interference functions, filtering the influence of interference noise synchronously during the super-resolution reconstruction process of the microwave image, avoiding the influence of the order of the two processes of super-resolution reconstruction and interference filtering on the quality of the super-resolution reconstructed image, and improving the efficiency of super-resolution reconstruction.

[0111] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not elaborate further. Those skilled in the art can understand that in the above methods of the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

[0112] In addition, the present disclosure also provides a neural network model training device, an anti-interference super-resolution reconstruction device for microwave images, an electronic device, and a non-volatile computer-readable storage medium. The above can all be used to implement any neural network model training method provided by the present disclosure, and / or the anti-interference super-resolution reconstruction method for microwave images. For the corresponding technical solutions and descriptions, please refer to the corresponding records in the method part and will not be elaborated further.

[0113] Figure 5 A block diagram showing a neural network model training device according to an embodiment of the present disclosure is as follows Figure 5 As shown, the device 500 includes:

[0114] The microwave image simulation module 501 is used to construct a second optical grayscale image set and a simulated microwave image set based on the first optical grayscale image set. Any simulated microwave image in the simulated microwave image set is simulated based on a second optical grayscale image in the second optical grayscale image set. Any second optical grayscale image in the second optical grayscale image set is downsampled from a first optical grayscale image in the first optical grayscale image set, and the resolution of the second optical grayscale image is lower than that of its corresponding first optical grayscale image.

[0115] The first network training module 502 is used to train a first neural network according to the first optical grayscale image set and the second optical grayscale image set. The first neural network is used to perform super-resolution processing on any optical grayscale image.

[0116] The second network training module 503 is used to determine an anti-interference super-resolution network through distillation training according to the first optical grayscale image set, the simulated microwave image set, and the first neural network. The anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used to perform anti-interference super-resolution reconstruction on any microwave image.

[0117] In a possible implementation manner, the microwave image simulation module 501 is specifically used to: for any first optical grayscale image in the first optical grayscale image set, perform downsampling processing on the first optical grayscale image to determine the second optical grayscale image corresponding to the first optical grayscale image; determine the second optical grayscale image set according to the second optical grayscale images corresponding to each first optical grayscale image in the first optical grayscale image set; for any second optical grayscale image in the second optical grayscale image set, perform speckle noise simulation on the second optical grayscale image to determine the simulated microwave image corresponding to the second optical grayscale image; determine the simulated microwave image set according to the simulated microwave images corresponding to each second optical grayscale image in the second optical grayscale image set.

[0118] In a possible implementation manner, the microwave image simulation module 501 is specifically used to: based on the gamma distribution, determine a multiplicative noise simulation function, where the multiplicative noise simulation function is used to simulate speckle noise; for any second optical grayscale image in the second optical grayscale image set, perform speckle noise simulation on the second optical grayscale image according to the multiplicative noise simulation function to determine the simulated microwave image corresponding to the second optical grayscale image.

[0119] In a possible implementation, the second network training module 503 is specifically configured to: input any second optical grayscale image in the second set of optical grayscale images into the first neural network to respectively determine the first encoded features corresponding to the first neural network and the super-resolution reconstructed image of the second optical grayscale image; input the simulated microwave image corresponding to the second optical grayscale image into the anti-interference super-resolution network to determine the second encoded features corresponding to the anti-interference super-resolution network and the super-resolution reconstructed image of the simulated microwave image; and iteratively train the anti-interference super-resolution network according to the first encoded features, the second encoded features, the super-resolution reconstructed image of the second optical grayscale image, the super-resolution reconstructed image of the simulated microwave image, and a preset multi-objective loss function until a preset training condition is reached, and determine the trained anti-interference super-resolution network.

[0120] In a possible implementation, the multi-objective loss function includes at least one of the following: reconstruction loss, feature distillation loss, target distillation loss, and gradient loss; wherein, the reconstruction loss is used to quantify the difference between the super-resolution reconstructed image obtained by the anti-interference super-resolution network for any simulated microwave image and the first optical grayscale image corresponding to the simulated microwave image; the feature distillation loss is used to quantify the feature distance between the first encoded features and the second encoded features; the target distillation loss is used to quantify the difference between the super-resolution reconstructed image obtained by the anti-interference super-resolution network for the simulated microwave image and the super-resolution reconstructed image obtained by the first neural network for the second optical grayscale image corresponding to the simulated microwave image; and the gradient loss is used to retain the texture details of the super-resolution reconstructed image output by the anti-interference super-resolution network.

[0121] Figure 6 The block diagram of an anti-interference super-resolution reconstruction device for microwave images according to an embodiment of the present disclosure is shown. As Figure 6 shown, the device 600 includes:

[0122] An image acquisition module 601, configured to determine a microwave image to be reconstructed;

[0123] A super-resolution reconstruction module 602, configured to input the microwave image to be reconstructed into the anti-interference super-resolution network to determine the super-resolution reconstructed image corresponding to the microwave image to be reconstructed, wherein the anti-interference super-resolution network is trained by the above method, and the resolution of the super-resolution reconstructed image is higher than the resolution of the microwave image to be reconstructed.

[0124] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0125] An embodiment of the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the above method.

[0126] An embodiment of the present disclosure also provides a non-volatile computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0127] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, the device 1900 may be provided as a server or a terminal device. Referring to Figure 7 , the device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0128] The device 1900 may also include a power component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0129] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, and the above computer program instructions can be executed by the processing component 1922 of the device 1900 to complete the above method.

[0130] A computer-readable storage medium can be a tangible device that can hold and store programs / instructions used by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being an instantaneous signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0131] The computer programs (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0132] A computer program (or computer program instructions) for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0133] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which instructions cause a computer, a programmable data processing apparatus, and / or other devices to operate in a particular manner, so that the computer-readable medium storing the instructions includes a manufacture comprising instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0137] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for training a neural network model, characterized in that, Including: Based on a first set of optical grayscale images, a second set of optical grayscale images and a set of simulated microwave images are respectively constructed. Among them, any one simulated microwave image in the set of simulated microwave images is simulated based on one second optical grayscale image in the second set of optical grayscale images. Any one second optical grayscale image in the second set of optical grayscale images is downsampled from one first optical grayscale image in the first set of optical grayscale images, and the resolution of this second optical grayscale image is lower than the resolution of its corresponding first optical grayscale image; According to the first set of optical grayscale images and the second set of optical grayscale images, a first neural network is trained, where the first neural network is used to perform super-resolution processing on any optical grayscale image; According to the first set of optical grayscale images, the set of simulated microwave images, and the first neural network, an anti-interference super-resolution network is determined through distillation training. Among them, the anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used to perform anti-interference super-resolution reconstruction on any microwave image.

2. The method according to claim 1, wherein The step of respectively constructing a second set of optical grayscale images and a set of simulated microwave images based on the first set of optical grayscale images includes: For any one first optical grayscale image in the first set of optical grayscale images, perform downsampling processing on this first optical grayscale image to determine the second optical grayscale image corresponding to this first optical grayscale image; According to the second optical grayscale images corresponding to each first optical grayscale image in the first set of optical grayscale images, determine the second set of optical grayscale images; For any one second optical grayscale image in the second set of optical grayscale images, perform speckle noise simulation on this second optical grayscale image to determine the simulated microwave image corresponding to this second optical grayscale image; According to the simulated microwave images corresponding to each second optical grayscale image in the second set of optical grayscale images, determine the set of simulated microwave images.

3. The method according to claim 2, wherein The step of performing speckle noise simulation on any one second optical grayscale image in the second set of optical grayscale images to determine the simulated microwave image corresponding to this second optical grayscale image includes: Based on the gamma distribution, determine a multiplicative noise simulation function, where the multiplicative noise simulation function is used to simulate speckle noise; For any one second optical grayscale image in the second set of optical grayscale images, perform speckle noise simulation on this second optical grayscale image according to the multiplicative noise simulation function to determine the simulated microwave image corresponding to this second optical grayscale image.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining the anti-interference super-resolution network through distillation training according to the first set of optical grayscale images, the set of simulated microwave images, and the first neural network includes: Input any one second optical grayscale image in the second set of optical grayscale images into the first neural network, and respectively determine the first encoded feature corresponding to the first neural network and the super-resolution reconstructed image of this second optical grayscale image; Input the simulated microwave image corresponding to the second optical grayscale image into the anti-interference super-resolution network to determine the second encoded feature corresponding to the anti-interference super-resolution network and the super-resolution reconstructed image of the simulated microwave image; Iteratively train the anti-interference super-resolution network according to the first encoded feature, the second encoded feature, the super-resolution reconstructed image of the second optical grayscale image, the super-resolution reconstructed image of the simulated microwave image, and a preset multi-objective loss function until a preset training condition is reached, and determine the trained anti-interference super-resolution network.

5. The method according to claim 4, wherein The multi-objective loss function includes at least one of the following: reconstruction loss, feature distillation loss, target distillation loss, and gradient loss; Among them, the reconstruction loss is used to quantify the difference between the super-resolution reconstructed image obtained by any simulated microwave image passing through the anti-interference super-resolution network and the first optical grayscale image corresponding to the simulated microwave image; The feature distillation loss is used to quantify the feature distance between the first encoded feature and the second encoded feature; The target distillation loss is used to quantify the difference between the super-resolution reconstructed image obtained by a simulated microwave image passing through the anti-interference super-resolution network and the super-resolution reconstructed image obtained by the second optical grayscale image corresponding to the simulated microwave image passing through the first neural network; The gradient loss is used to retain the texture details of the super-resolution reconstructed image output by the anti-interference super-resolution network.

6. A method for anti-interference super-resolution reconstruction of microwave images, characterized in that, Includes: Determine the microwave image to be reconstructed; Input the microwave image to be reconstructed into the anti-interference super-resolution network to determine the super-resolution reconstructed image corresponding to the microwave image to be reconstructed, where the anti-interference super-resolution network is trained by the method according to any one of claims 1 to 5, and the resolution of the super-resolution reconstructed image is higher than the resolution of the microwave image to be reconstructed.

7. A neural network model training device, characterized in that, Includes: A microwave image simulation module for respectively constructing a second optical grayscale image set and a simulated microwave image set based on a first optical grayscale image set, where any simulated microwave image in the simulated microwave image set is simulated based on a second optical grayscale image in the second optical grayscale image set, and any second optical grayscale image in the second optical grayscale image set is downsampled from a first optical grayscale image in the first optical grayscale image set, and the resolution of the second optical grayscale image is lower than the resolution of its corresponding first optical grayscale image; A first network training module for training a first neural network according to the first optical grayscale image set and the second optical grayscale image set, where the first neural network is used for super-resolution processing of any optical grayscale image; A second network training module, configured to determine an anti-interference super-resolution network through distillation training according to the first optical grayscale image set, the simulated microwave image set, and the first neural network, wherein the anti-interference super-resolution network has the same network structure as the first neural network, and the anti-interference super-resolution network is used to perform anti-interference super-resolution reconstruction on any microwave image.

8. An anti-interference super-resolution reconstruction device for microwave images, characterized in that, Comprising: An image acquisition module, configured to determine a microwave image to be reconstructed; A super-resolution reconstruction module, configured to input the microwave image to be reconstructed into the anti-interference super-resolution network to determine a super-resolution reconstruction image corresponding to the microwave image to be reconstructed, wherein the anti-interference super-resolution network is trained by the method according to any one of claims 1 to 5, and the resolution of the super-resolution reconstruction image is higher than that of the microwave image to be reconstructed.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images