A radar microwave image data set generation method, device and electronic equipment

Abstract

The embodiments of this specification disclose a radar microwave image data set generation method, device and electronic device. The method includes constructing and training a bidirectional conversion deep learning model, and then converting a large batch of optical image data sets to be converted through the trained bidirectional conversion deep learning model to obtain a large batch of microwave image data sets that are severely missing. In the above deep learning generation model method, by setting a forward generative adversarial network model and a reverse generative adversarial network model in the bidirectional conversion deep learning model, and setting a distortion adaptive function to continuously adjust the deep learning model parameters, the finally generated radar microwave image data set has high fidelity, rich details, and realistic structure, which enhances the consistency and authenticity of the corresponding optical images and microwave images at different viewing angles, and meets the subsequent deep learning requirements for large batches of high-quality training samples in the intelligent application of spaceborne synthetic aperture radar.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of synthetic aperture radar satellite remote sensing technology, and in particular, to a method, device and electronic equipment for generating a radar microwave image data set. Background Art

[0002] Compared with optical remote sensing satellites with a visibility of about 25%, synthetic aperture radar (SAR) satellites have 100% visibility due to their all-weather and all-day earth observation capabilities, and have shown irreplaceable importance and great application potential in the fields of disaster reduction and prevention. However, the imaging mechanism of SAR satellite microwave images (hereinafter referred to as microwave images) is complex, which is essentially different from optical images. Artificial intelligence deep learning technology is needed to interpret their information, but the microwave training samples required for deep learning model training are seriously insufficient, which hinders the smooth development and urgent needs of automatic interpretation of SAR remote sensing images. Therefore, a method for generating microwave image datasets from easily accessible satellite remote sensing optical images (hereinafter referred to as optical images) is needed to obtain a large number of microwave training samples. The existing common method is to use image conversion generation models such as Pix2Pix to convert a large number of optical images into large microwave image datasets for deep learning model training of target recognition. However, due to the different modalities of optical images and microwave images, conversion generative models such as Pix2Pix have shortcomings in the multimodal conversion of optical images to microwave images, such as limited generalization ability and mode collapse problems caused by sensitivity to noise and outliers. They cannot guarantee the fidelity of the target characteristics of the converted image, making it difficult for the generated microwave image dataset to meet the high-precision deep learning model training requirements of key applications such as target recognition. Summary of the invention

[0003] The embodiments of this specification provide a radar microwave image data set generation method, device and electronic device, and the technical solution thereof is as follows:

[0004] In a first aspect, an embodiment of this specification provides a method for generating a radar microwave image dataset, the method comprising:

[0005] Constructing an initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in both the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion;

[0006] Acquire an optical image data set to be converted, and for any optical image in the optical image data set, convert the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image;

[0007] All the microwave images obtained are integrated to obtain a microwave image dataset.

[0008] In a second aspect, a radar microwave image data set generation device is provided, the device comprising:

[0009] A construction module, for constructing an initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in both the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion;

[0010] A conversion module, used for acquiring an optical image data set to be converted, and for any optical image in the optical image data set, converting the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image;

[0011] The integration module is used to integrate all the microwave images obtained to obtain a microwave image data set.

[0012] In a third aspect, an electronic device is provided, including a device processor and a memory;

[0013] The device processor is connected to the memory;

[0014] The memory is used to store executable program code;

[0015] The device processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method provided in the first aspect or any possible implementation manner of the first aspect.

[0016] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and the computer-readable storage medium stores instructions, which, when the instructions are executed on a computer or a device processor, cause the computer or the device processor to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0017] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0018] In one or more embodiments of the present specification, by constructing an initial bidirectional conversion deep learning model framework and algorithm, the initial bidirectional conversion deep learning model is trained based on a smaller batch of "optical-microwave" paired original image sets to obtain a trained bidirectional conversion deep learning model, and then the large batch of optical image data sets to be converted are converted through the trained bidirectional conversion deep learning model to obtain microwave images, and finally all the obtained microwave images are integrated to obtain a large batch of microwave image data sets. The method of obtaining microwave images through the above deep learning model not only improves the efficiency of generating large microwave image data sets, but also continuously adjusts the deep learning model parameters by setting a forward generative adversarial network model and a reverse generative adversarial network model in the bidirectional conversion deep learning model, and setting a distortion adaptive function, so that the microwave image data set finally generated by the bidirectional conversion deep learning model has high fidelity, rich details, and realistic structure, and enhances the consistency and authenticity of the corresponding optical images and microwave images at different viewing angles, and meets the demand for large batches of high-quality microwave image samples in the deep learning model training required in the key applications of subsequent microwave image intelligent interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of the system architecture of a radar microwave image data set generation method provided in an embodiment of this specification;

[0021] Figure 2 A flowchart of a radar microwave image data set generation method provided in an embodiment of this specification;

[0022] Figure 3 A schematic diagram of the structure of a radar microwave image data set generation device provided in an embodiment of this specification;

[0023] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0025] The terms "first", "second", "third", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0026] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present specification. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described with respect to some examples may be combined in other examples.

[0027] See also Figure 1 , Figure 1 A schematic diagram of the system architecture of a radar microwave image data set generation method provided in an embodiment of this specification is shown.

[0028] like Figure 1 As shown, the system architecture of the radar microwave image data set generation method may include at least a terminal 10 , a server 20 and a network 30 .

[0029] The terminal 10 includes but is not limited to electronic devices such as smart phones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, smart wearable devices, etc., and may also be software running on the above electronic devices, such as applications, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc. Optionally, the terminal 10 provides a data set generation service to the user, and the terminal 10 may obtain a data set generation instruction of an application program interface and send a data set generation request to the server 20.

[0030] The server 20 can provide background services for the terminal 10. According to the data set generation request sent by the terminal 10, the server 20 will obtain a series of data set generation instructions, and the server 20 transmits the data set generation instructions to other terminals 10 through the network 30. Specifically, the server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0031] The network 30 is used to provide a medium for a communication link between the terminal 10 and the server 20. The network 30 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0032] In addition, it should be noted that Figure 1 What is shown is only one system provided by the present disclosure. In practical applications, other systems may also be included, for example, more terminals may be included.

[0033] In the embodiments of the present specification, the terminal 10 and the server 20 may be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure.

[0034] See next Figure 2 , Figure 2 The overall flow chart of a radar microwave image dataset generation method provided in an embodiment of the present specification is shown. The radar microwave image dataset generation method can be used in the server 20 .

[0035] like Figure 2As shown, the radar microwave image data set generation method may at least include the following steps:

[0036] Step 201: construct an initial bidirectional conversion deep learning model, and train the initial bidirectional conversion deep learning model based on the original image set to obtain a trained bidirectional conversion deep learning model.

[0037] Among them, the initial bidirectional deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, the original image set includes original optical images and original microwave images that are paired in pairs, and the forward generative adversarial network model and the reverse generative adversarial network model are both provided with a distortion adaptive function, and the distortion adaptive function is used to adjust model parameters to minimize the pixel error between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion.

[0038] In the embodiment of the present specification, a forward generative adversarial network model and a reverse generative adversarial network model can be designed and constructed first, and the two adversarial network models are synthesized to construct an initial bidirectional conversion deep learning model. Then, the original optical image and the original microwave image, that is, the original image set, are obtained in pairs. Further, the original optical image is converted at least once through the constructed forward generative adversarial network model and the reverse generative adversarial network model, and the test microwave image and the test optical image are obtained respectively. As an example, when it is selected to undergo only one image conversion, the original optical image can be obtained by passing the forward generative adversarial network model to obtain the required test microwave image, and then the obtained test microwave image is reversely input into the reverse generative adversarial network model to obtain the required test optical image. When it is selected to undergo multiple adversarial network model conversions, the image obtained by the last cyclic conversion is used as the test microwave image and the test optical image. Then, the pixel error between the original optical image and the test optical image, and the pixel error between the original microwave image and the test microwave image are determined. When the two pixel errors obtained cannot meet the minimization requirements, the model parameters of the bidirectional conversion deep learning model are continuously adjusted by the distortion adaptive function until the two pixel errors can meet the minimization requirements.

[0039] In one possible implementation, the constructing of an initial bidirectional conversion deep learning model includes:

[0040] Construct forward generative adversarial network model and reverse generative adversarial network model based on the bidirectional closed-loop method;

[0041] The forward generative adversarial network model and the reverse generative adversarial network model are combined based on the dual discriminator structure to obtain an initial bidirectional conversion deep learning model.

[0042] In the embodiment of this specification, a forward generative adversarial network model and a reverse generative adversarial network model are first constructed by bidirectional closed-loop generation technology, and the forward generative adversarial network model can convert the optical image input thereinto into a microwave image, and the reverse generative adversarial network model can convert the microwave image input thereinto into an optical image. Then, the constructed forward generative adversarial network model and the reverse generative adversarial network model are combined through a dual discriminator structure, so that the output of one model can be used as the input of the other model.

[0043] In one possible implementation, the initial bidirectional conversion deep learning model is trained based on the original image set to obtain a trained bidirectional conversion deep learning model, including:

[0044] Inputting the original optical image in the original image set into the forward generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test microwave image;

[0045] Inputting the first test microwave image into the reverse generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test optical image;

[0046] Inputting the first test optical image into the forward generative adversarial network model to obtain a second test microwave image;

[0047] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0048] In an embodiment of the present specification, when the initial bidirectional conversion deep learning model is trained, the original optical image in the original image set can be first input into the forward generative adversarial network model in the initial bidirectional conversion deep learning model, and the first test microwave image is obtained by forward conversion. Then, the obtained first test microwave image is input into the reverse generative adversarial network model in the initial bidirectional conversion deep learning model, and the first test optical image is obtained by reverse conversion; further, in order to improve the training effect of the bidirectional conversion deep learning model, it is necessary to input the obtained first test optical image into the forward generative adversarial network model to obtain the second test microwave image. Finally, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted by the distortion adaptive function until the pixel errors between the first test optical image and the original optical image, and the second test microwave image and the original microwave image meet the preset requirements, and then the parameters in each adversarial network model are fixed at this time to obtain a trained bidirectional conversion deep learning model.

[0049] Among them, when the pixel error between the first test optical image and the original optical image does not meet the preset requirements, it is necessary to adjust the model parameters in the reverse generative adversarial network model; when the pixel error between the second test microwave image and the original microwave image does not meet the preset requirements, it is necessary to adjust the model parameters in the forward generative adversarial network model.

[0050] In one possible implementation, the method continuously adjusts each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet preset requirements, thereby obtaining a trained bidirectional conversion deep learning model, including:

[0051] determining a first pixel error between the first test optical image and the original optical image, and determining a second pixel error between the second test microwave image and the original microwave image;

[0052] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the first pixel error and the second pixel error both meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0053] In an embodiment of the present specification, when the model parameters are continuously adjusted by the image pixel error, the first pixel error between the first test optical image and the original optical image, and the second pixel error between the second test microwave image and the original microwave image can be determined respectively. Then, it is determined whether the obtained first pixel error and the second pixel error meet the preset requirements. When only one of the first pixel error and the second pixel error does not meet the preset requirements, it is necessary to adjust the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model through the distortion adaptive function. Among them, the model parameters of the forward generative adversarial network model are used as the adjustment basis, and the second pixel error is first made to meet the preset requirements, and then the model parameters in the reverse generative adversarial network model are further adjusted. Finally, when it is determined that the first pixel error and the second pixel error all meet the preset requirements, the adjustment of each parameter is stopped to obtain a trained bidirectional conversion deep learning model.

[0054] In one possible implementation, the method of continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until both the first pixel error and the second pixel error meet preset requirements includes:

[0055] Continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function to obtain the first pixel error and the second pixel error;

[0056] Comparing the first pixel error and the second pixel error with a preset minimum error respectively, to obtain a first comparison result corresponding to the first pixel error and a second comparison result corresponding to the second pixel error;

[0057] When the first comparison result and the second comparison result are both characterized as being smaller than the preset minimum error, the first pixel error and the second pixel error are characterized as both satisfying preset requirements.

[0058] In the embodiment of the present specification, when judging whether the first pixel error and the second pixel error both meet the preset requirements, the model parameters of the forward generative adversarial network model can be used as the adjustment reference through the distortion adaptive function, and the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model can be adjusted once to obtain a set of pixel error parameters, that is, including the first pixel error and the second pixel error. Then, the first pixel error is compared with the preset minimum error to obtain a first comparison result, and the second pixel error is compared with the preset minimum error by difference to obtain a second comparison result. Further, when there is only one non-negative number in the first comparison result and the second comparison result, that is, the pixel error corresponding to at least one comparison result is not less than the preset minimum error, that is, the preset requirement of minimizing the error is not met, and it is necessary to readjust the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model according to the error size and direction, until the first comparison result and the second comparison result are all negative numbers, that is, the pixel errors corresponding to all comparison results are less than the preset minimum error, and the parameter adjustment is stopped.

[0059] In one possible implementation, the constructing of the initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on the original image set, and obtaining the trained bidirectional conversion deep learning model further includes:

[0060] Acquire a verification image with target characteristics, and input the target optical image in the verification image into the trained bidirectional conversion deep learning model to obtain a verification microwave image;

[0061] determining a target distortion between the verification microwave image and a target microwave image in the verification image;

[0062] Based on the feedback adjustment method, a feedback adjustment parameter corresponding to the target distortion is determined, and based on the feedback adjustment parameter, the trained bidirectional conversion deep learning model is adjusted to obtain an adjusted bidirectional conversion deep learning model.

[0063] In the embodiment of the present specification, in order to improve the application accuracy of the bidirectional conversion deep learning model, after obtaining the trained bidirectional conversion deep learning model, a verification image with target characteristics can be obtained. Among them, the verification image may include a target optical image and a target microwave image that are paired in pairs. Then, the target optical image is input into the trained bidirectional conversion deep learning model to obtain a verification microwave image, and the target distortion between the verification microwave image obtained by positive conversion and the target microwave image is determined. Further, the distortion range of the target distortion is first determined by the feedback adjustment method. If the target distortion belongs to the acceptable distortion range, the corresponding feedback adjustment parameter is 0. When the target distortion exceeds the distortion range, the corresponding feedback adjustment parameter can be found in the distortion adjustment parameter database, and the trained bidirectional conversion deep learning model is adjusted twice by the feedback adjustment parameter to obtain the adjusted bidirectional conversion deep learning model.

[0064] Step 202: Acquire an optical image data set to be converted, and for any optical image in the optical image data set, convert the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image.

[0065] In an embodiment of the present specification, in order to generate a microwave image data set, after obtaining a trained bidirectional conversion deep learning model, an optical image data set to be converted can be obtained, and each optical image of the optical image data set can be converted through a forward generative adversarial network model in the trained bidirectional conversion deep learning model to obtain a microwave image.

[0066] Among them, according to the actual microwave image accuracy requirements, the acquired optical images can also be subjected to multiple cyclic bidirectional image conversions in the trained bidirectional conversion deep learning model in the same principle as the model training process to obtain microwave images that meet the requirements.

[0067] Step 203: Integrate all the microwave images obtained to obtain a microwave image data set.

[0068] In an embodiment of the present specification, after the microwave images corresponding to all the optical images are obtained by using the trained bidirectional conversion deep learning model, all the microwave images are statistically sorted and merged to obtain a microwave image data set.

[0069] In one possible implementation, the integrating all the obtained microwave images to obtain a microwave image dataset includes:

[0070] Acquire all the microwave images, and screen out all the microwave images based on the target characteristic threshold to obtain qualified microwave images;

[0071] All the qualified microwave images are integrated to obtain a microwave image data set.

[0072] In the embodiment of the present specification, in order to improve the sample quality of the microwave image data set finally obtained, the target characteristic screening of the obtained microwave images can be performed. First, all the microwave images after conversion can be obtained. Then, the target characteristic value of each microwave image is determined by the image feature value calculation method, and the target characteristic value is compared with the preset target characteristic threshold value, and the microwave images whose target characteristics are less than the target characteristic threshold value and are not obvious are screened out, that is, each qualified microwave image is obtained. Finally, all qualified microwave images whose target characteristics meet the requirements are integrated to obtain a microwave image data set.

[0073] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] See next Figure 3 , Figure 3 FIG. 1 is a schematic diagram showing a structure of a radar microwave image data set generation device provided in an embodiment of the present specification. It should be noted that: Figure 3 The radar microwave image data set generation device shown is used to execute the present application Figure 2 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2 The embodiment shown.

[0075] like Figure 3 As shown, the radar microwave image data set generating device may at least include:

[0076] A construction module 301 is used to construct an initial bidirectional conversion deep learning model, and train the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion;

[0077] A conversion module 302 is used to obtain an optical image data set to be converted, and for any optical image in the optical image data set, convert the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image;

[0078] The integration module 303 is used to integrate all the obtained microwave images to obtain a microwave image data set.

[0079] In one possible implementation, the construction module 301 is specifically used for:

[0080] Construct forward generative adversarial network model and reverse generative adversarial network model based on the bidirectional closed-loop method;

[0081] The forward generative adversarial network model and the reverse generative adversarial network model are combined based on the dual discriminator structure to obtain an initial bidirectional conversion deep learning model.

[0082] In one possible implementation, the construction module 301 is further specifically used for:

[0083] Inputting the original optical image in the original image set into the forward generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test microwave image;

[0084] Inputting the first test microwave image into the reverse generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test optical image;

[0085] Inputting the first test optical image into the forward generative adversarial network model to obtain a second test microwave image;

[0086] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0087] In one possible implementation, the construction module 301 is further specifically used for:

[0088] determining a first pixel error between the first test optical image and the original optical image, and determining a second pixel error between the second test microwave image and the original microwave image;

[0089] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the first pixel error and the second pixel error both meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0090] In one possible implementation, the construction module 301 is further specifically used for:

[0091] Continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function to obtain the first pixel error and the second pixel error;

[0092] Comparing the first pixel error and the second pixel error with a preset minimum error respectively, to obtain a first comparison result corresponding to the first pixel error and a second comparison result corresponding to the second pixel error;

[0093] When the first comparison result and the second comparison result are both characterized as being smaller than the preset minimum error, the first pixel error and the second pixel error are characterized as both satisfying preset requirements.

[0094] In one possible implementation, the construction module 301 is further specifically used for:

[0095] Acquire a verification image with target characteristics, and input the target optical image in the verification image into the trained bidirectional conversion deep learning model to obtain a verification microwave image;

[0096] determining a target distortion between the verification microwave image and a target microwave image in the verification image;

[0097] Based on the feedback adjustment method, a feedback adjustment parameter corresponding to the target distortion is determined, and based on the feedback adjustment parameter, the trained bidirectional conversion deep learning model is adjusted to obtain an adjusted bidirectional conversion deep learning model.

[0098] In one possible implementation, the integration module 303 is further configured to:

[0099] Acquire all the microwave images, and screen out all the microwave images based on the target characteristic threshold to obtain qualified microwave images;

[0100] All the qualified microwave images are integrated to obtain a microwave image data set.

[0101] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. The "unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a high-performance computer server, workstation or dedicated computing device equipped with a necessary graphics processing unit (GPU).

[0102] Each processing unit and / or module of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.

[0103] See next Figure 4 , Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification is shown.

[0104] like Figure 4 As shown, the electronic device 400 may include: at least one device processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0105] The communication bus 402 may be used to realize the connection and communication among the above-mentioned components.

[0106] The user interface 403 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0107] The network interface 404 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0108] Among them, the device processor 401 may include one or more processing cores. The device processor 401 uses various interfaces and lines to connect various parts within the entire electronic device 400, and executes various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. The device processor 401 can integrate one or a combination of CPU, GPU and modem, etc. Among them, the CPU mainly processes the operating system, user interface and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the device processor 401, and it can be implemented separately through a chip.

[0109] The memory 405 may include a RAM or a ROM. Optionally, the memory 405 includes a non-transitory computer-readable medium. The memory 405 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 405 may optionally be at least one storage device located away from the aforementioned device processor 401. As Figure 4 As shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0110] Specifically, the device processor 401 may be used to call the radar microwave image data set generation application stored in the memory 405, and specifically perform the following operations:

[0111] Constructing an initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in both the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion;

[0112] Acquire an optical image data set to be converted, and for any optical image in the optical image data set, convert the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image;

[0113] All the microwave images obtained are integrated to obtain a microwave image dataset.

[0114] As an option of the embodiment of this specification, the constructing of the initial bidirectional conversion deep learning model includes:

[0115] Construct forward generative adversarial network model and reverse generative adversarial network model based on the bidirectional closed-loop method;

[0116] The forward generative adversarial network model and the reverse generative adversarial network model are combined based on the dual discriminator structure to obtain an initial bidirectional conversion deep learning model.

[0117] As an option of the embodiments of this specification, the initial bidirectional conversion deep learning model is trained based on the original image set to obtain a trained bidirectional conversion deep learning model, including:

[0118] Inputting the original optical image in the original image set into the forward generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test microwave image;

[0119] Inputting the first test microwave image into the reverse generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test optical image;

[0120] Inputting the first test optical image into the forward generative adversarial network model to obtain a second test microwave image;

[0121] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0122] As an optional embodiment of the present specification, the method continuously adjusts each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, and obtains a trained bidirectional conversion deep learning model, including:

[0123] determining a first pixel error between the first test optical image and the original optical image, and determining a second pixel error between the second test microwave image and the original microwave image;

[0124] Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the first pixel error and the second pixel error both meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

[0125] As an optional embodiment of the present specification, the method of continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until both the first pixel error and the second pixel error meet the preset requirements includes:

[0126] Continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function to obtain the first pixel error and the second pixel error;

[0127] Comparing the first pixel error and the second pixel error with a preset minimum error respectively, to obtain a first comparison result corresponding to the first pixel error and a second comparison result corresponding to the second pixel error;

[0128] When the first comparison result and the second comparison result are both characterized as being smaller than the preset minimum error, the first pixel error and the second pixel error are characterized as both satisfying preset requirements.

[0129] As an optional embodiment of the present specification, the constructing of the initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on the original image set, and obtaining the trained bidirectional conversion deep learning model further includes:

[0130] Acquire a verification image with target characteristics, and input the target optical image in the verification image into the trained bidirectional conversion deep learning model to obtain a verification microwave image;

[0131] determining a target distortion between the verification microwave image and a target microwave image in the verification image;

[0132] Based on the feedback adjustment method, a feedback adjustment parameter corresponding to the target distortion is determined, and based on the feedback adjustment parameter, the trained bidirectional conversion deep learning model is adjusted to obtain an adjusted bidirectional conversion deep learning model.

[0133] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0134] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0135] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0140] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct related hardware, and the program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0141] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for generating a radar microwave image data set, characterized in that: The method comprises: Constructing an initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional conversion deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in both the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion; Acquire an optical image data set to be converted, and for any optical image in the optical image data set, convert the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image; Integrating all the obtained microwave images to obtain a microwave image data set; The constructing of the initial bidirectional conversion deep learning model includes: Construct forward generative adversarial network model and reverse generative adversarial network model based on the bidirectional closed-loop method; Combining the forward generative adversarial network model and the reverse generative adversarial network model based on a dual discriminator structure to obtain an initial bidirectional conversion deep learning model; The constructed forward generative adversarial network model and the reverse generative adversarial network model are combined through a dual discriminator structure, so that the output of one model serves as the input of the other model.

2. The method according to claim 1, characterized in that The initial bidirectional conversion deep learning model is trained based on the original image set to obtain a trained bidirectional conversion deep learning model, including: Inputting the original optical image in the original image set into the forward generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test microwave image; Inputting the first test microwave image into the reverse generative adversarial network model in the initial bidirectional conversion deep learning model to obtain a first test optical image; Inputting the first test optical image into the forward generative adversarial network model to obtain a second test microwave image; Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

3. The method according to claim 2, characterized in that The method continuously adjusts the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until the pixel errors between the first test optical image and the original optical image, and between the second test microwave image and the original microwave image meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model, including: determining a first pixel error between the first test optical image and the original optical image, and determining a second pixel error between the second test microwave image and the original microwave image; Based on the distortion adaptive function, the model parameters in the forward generative adversarial network model and the reverse generative adversarial network model are continuously adjusted until the first pixel error and the second pixel error both meet the preset requirements, thereby obtaining a trained bidirectional conversion deep learning model.

4. The method according to claim 3, characterized in that The method of continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function until both the first pixel error and the second pixel error meet preset requirements includes: Continuously adjusting each model parameter in the forward generative adversarial network model and the reverse generative adversarial network model based on the distortion adaptive function to obtain the first pixel error and the second pixel error; Comparing the first pixel error and the second pixel error with a preset minimum error respectively, to obtain a first comparison result corresponding to the first pixel error and a second comparison result corresponding to the second pixel error; When the first comparison result and the second comparison result are both characterized as being smaller than the preset minimum error, the first pixel error and the second pixel error are characterized as both satisfying preset requirements.

5. The method according to claim 1, characterized in that After constructing the initial bidirectional conversion deep learning model and training the initial bidirectional conversion deep learning model based on the original image set to obtain the trained bidirectional conversion deep learning model, the method further includes: Acquire a verification image with target characteristics, and input the target optical image in the verification image into the trained bidirectional conversion deep learning model to obtain a verification microwave image; determining a target distortion between the verification microwave image and a target microwave image in the verification image; Based on the feedback adjustment method, a feedback adjustment parameter corresponding to the target distortion is determined, and based on the feedback adjustment parameter, the trained bidirectional conversion deep learning model is adjusted to obtain an adjusted bidirectional conversion deep learning model.

6. The method according to claim 1, characterized in that The step of integrating all the microwave images to obtain a microwave image dataset includes: Acquire all the microwave images, and screen out all the microwave images based on the target characteristic threshold to obtain qualified microwave images; All the qualified microwave images are integrated to obtain a microwave image data set.

7. A radar microwave image data set generation device, characterized in that: The device comprises: A construction module, for constructing an initial bidirectional conversion deep learning model, training the initial bidirectional conversion deep learning model based on an original image set to obtain a trained bidirectional conversion deep learning model, wherein the initial bidirectional conversion deep learning model includes a forward generative adversarial network model and a reverse generative adversarial network model, wherein the forward generative adversarial network model is used to convert an optical image into a microwave image, and the reverse generative adversarial network model is used to convert a microwave image into an optical image, wherein the original image set includes original optical images and original microwave images that are paired in pairs, respectively, and a distortion adaptive function is provided in both the forward generative adversarial network model and the reverse generative adversarial network model, wherein the distortion adaptive function is used to adjust model parameters to minimize pixel errors between the original optical image and a test optical image obtained after at least one conversion, and between the original microwave image and a test microwave image obtained after at least one conversion; A conversion module, used for acquiring an optical image data set to be converted, and for any optical image in the optical image data set, converting the optical image based on the trained bidirectional conversion deep learning model to obtain a microwave image; An integration module, used for integrating all the microwave images obtained to obtain a microwave image data set; The building blocks are specifically used to: Construct forward generative adversarial network model and reverse generative adversarial network model based on the bidirectional closed-loop method; Combining the forward generative adversarial network model and the reverse generative adversarial network model based on a dual discriminator structure to obtain an initial bidirectional conversion deep learning model; The constructed forward generative adversarial network model and the reverse generative adversarial network model are combined through a dual discriminator structure, so that the output of one model serves as the input of the other model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, wherein the computer-readable storage medium has instructions stored therein, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 6.

Images