Multi-aspect sample expansion method for air target ISAR imaging based on deep learning

Abstract

The application provides a deep learning-based multi-aspect sample expansion method for ISAR imaging of air targets, belongs to the field of ISAR imaging information acquisition and target identification of air targets, and is used for expanding samples in the aspect dimension of air targets.The application comprises the following steps: step S1, acquiring an ISAR image simulation dataset of an air target by modeling the air target and fast electromagnetic scattering calculation; step S2, dividing the ISAR image simulation dataset of the air target by setting interval degrees to establish a training set and a test set; step S3, building a deep learning network model having at least a generator and a discriminator; step S4, inputting data with a model category label in the training set into the deep learning network model to train the generator and the discriminator; and step S5, inputting data in the test set into the trained deep learning network model to output images under corresponding information.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of ISAR imaging information acquisition and target recognition of air targets, and particularly relates to an ISAR imaging multi-aspect sample expansion method for air targets based on deep learning. BACKGROUND

[0002] In the field of target imaging and recognition, SAR / ISAR imaging technology has advantages such as all-weather and all-day imaging, and has become an important means of target detection and recognition. SAR is commonly used for imaging of moving targets by a flying sensor, and ISAR is commonly used for imaging of moving targets by a fixed radar.

[0003] With the development of artificial intelligence and machine learning theories and methods, intelligent target recognition technology has excellent recognition ability under the training of a large number of samples. However, in actual application, sample acquisition is difficult and expensive in many application scenarios, resulting in a very small amount of samples that can be used for training, and not enough samples to train the target recognition model. For example, it is usually difficult to obtain a large number of ISAR images of aircraft and their corresponding models, resulting in a lack of ISAR samples of air targets. For air targets, this lack mainly manifests in the lack of samples in the angle dimension.

[0004] With the research and application boom of a new generation of artificial intelligence technology triggered by deep learning technology in recent years, humans have entered the intelligent era, and deep learning has developed rapidly in applications such as optical image classification, target detection and recognition, and has gradually replaced humans to become the main force in various image recognition tasks. In the field of deep learning, there are some methods for dealing with the problem of small sample target classification and recognition, such as meta-learning-based transfer learning methods, supervised learning-based methods and sample expansion methods based on generative network models. Among them, the sample expansion methods based on generative network models include conventional sample expansion methods such as rotation, translation and brightness transformation. However, few deep learning methods are used for the problem of sample expansion in the azimuth dimension of air target ISAR images to solve the problem of sample lack in the azimuth dimension of air targets. SUMMARY

[0005] To solve the above problems, an ISAR imaging multi-aspect sample expansion method for air targets based on deep learning is provided for expanding samples in the azimuth dimension of air targets. The application adopts the following technical solutions:

[0006] The application provides an air target ISAR imaging multi-aspect sample expansion method based on deep learning, and has the characteristics that the method comprises the following steps: step S1, obtaining a full-aspect air ISAR image simulation dataset by modeling an air target and performing fast electromagnetic scattering calculation; step S2, dividing the air ISAR image simulation dataset by setting interval degrees to establish a training set and a test set; step S3, building a deep learning network model, wherein the deep learning network model at least has a generator for generating data and a discriminator for constraining the generation result of the generator; step S4, inputting data with a model category label in the training set into the deep learning network model to train the generator and the discriminator of the deep learning network model; and step S5, inputting data in the test set into the trained deep learning network model to output an image under corresponding information.

[0007] The air target ISAR imaging multi-aspect sample expansion method based on deep learning provided by the application can further have the following characteristics: step S1 comprises the following sub-steps: step S1-1, establishing an air target model with equal proportions and sizes according to the specific size of the air target; step S1-2, performing triangular facet division on the air target model to make the facet size meet the imaging frequency band requirement; step S1-3, performing electromagnetic simulation calculation on the air target model after triangular facet division to obtain RCS echo data according to the set imaging parameters; and step S1-4, performing imaging on the RCS echo data to obtain a full-aspect air target ISAR image simulation dataset.

[0008] The air target ISAR imaging multi-aspect sample expansion method based on deep learning provided by the application can further have the following characteristics: step S2 extracts the full-aspect air ISAR image simulation dataset by setting interval degrees, takes the extracted air target ISAR image simulation dataset as the training set, and takes the unextracted dataset as the test set.

[0009] The air target ISAR imaging multi-aspect sample expansion method based on deep learning provided by the application can further have the following characteristics: the deep learning network model in step S3 is in the form of a conditional generative adversarial network framework, the generator and the discriminator both comprise a convolution layer, a Batch-Norm layer and an activation function layer, and the generator and the discriminator work alternately.

[0010] The deep learning-based multi-aspect sample expansion method for ISAR imaging of air targets provided by the application can further have the following features: step S4 comprises the following sub-steps: step S4-1, inputting training random noise, training categories and training angles in the training set into the input end of the generator to generate an ISAR image of an air target under corresponding information, step S4-2, the discriminator discriminates the fidelity of the generated ISAR image of the air target under the corresponding information, and judges whether the expansion data generated by the generator conforms to the input training angle and training category, to constrain the generation result of the generator, and step S4-3, calculating the total loss function of the deep learning network model, and the calculation formula is as follows:

[0011]

[0012] Wherein v = [cosφ, sinφ] T represents an angle, c represents a category, and z represents random noise.

[0013] The deep learning-based multi-aspect sample expansion method for ISAR imaging of air targets provided by the application can further have the following features: step S5 comprises the following sub-steps: step S5-1, inputting test angles and test categories in the test set into the input end of the trained generator to obtain an ISAR image of an air target under corresponding information, and step S5-2, comparing the generated ISAR image of the air target under the corresponding information with the test set in step S2 to verify the generation effect.

[0014] Invention action and effect

[0015] The deep learning-based multi-aspect sample expansion method for ISAR imaging of air targets according to the application can make the intelligent target recognition technology have sufficient recognition ability in the case that there are few samples in the field of ISAR target sample recognition of air targets. The multi-aspect sample expansion method of the application can still have a high recognition rate under the condition of small samples, and can assist or replace artificial work in many application fields. It can be seen that the method solves the recognition problem of air targets under the condition of small samples, and solves the problem of missing aspect samples of air targets by using the sample expansion method based on the generated deep learning network model. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a process diagram of fine modeling and fast electromagnetic imaging calculation of an air target in an embodiment of the application;

[0017] Figure 2 is a structure diagram of 3D modeling of Airbus-321 in an embodiment of the application;

[0018] Figure 3is a sectional view of a 3D image of Airbus-321 in an embodiment of the present application;

[0019] Figure 4 is a full-coverage ISAR image schematic diagram of Airbus-321 in an embodiment of the present application;

[0020] Figure 5 is a full-coverage ISAR image schematic diagram of Airbus-350 in an embodiment of the present application;

[0021] Figure 6 is a structure diagram of a deep learning network model built in an embodiment of the present application;

[0022] Figure 7 is a generator structure diagram for gradually improving image resolution in an embodiment of the present application;

[0023] Figure 8 is a process diagram of a generator based on the built deep learning network model for generating a final image in an embodiment of the present application;

[0024] Figure 9 is a generated result image in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The specific embodiments of the present application will be described below in conjunction with the accompanying drawings and embodiments.

[0026] <EMBODIMENT>

[0027] The embodiment provides a deep learning-based multi-azimuth sample expansion method for ISAR imaging of an air target, which is used for improving the air target recognition rate under the condition of a small sample and avoiding the lack of an air target azimuth dimension sample.

[0028] Figures 1 to 9 is a total flow of the deep learning-based multi-azimuth sample expansion method for ISAR imaging of an air target in an embodiment of the present application.

[0029] The deep learning-based multi-azimuth sample expansion method for ISAR imaging of an air target in the embodiment of the present application will be described below. Figures 1 to 9 Step S1, fine modeling of the air target is performed and fast electromagnetic imaging calculation is performed.

[0030] Step S1 includes the following sub-steps:

[0031]

[0032] Figure 1 is a process diagram of fine modeling and fast electromagnetic imaging calculation of an air target in an embodiment of the present application; Figure 2 is a structure diagram of 3D modeling of Airbus-321 in an embodiment of the present application;​Figure 3 is a schematic diagram of the sectioning of the 3D image of Airbus-321 in the embodiment of the present application.

[0033] As shown in Figures 1 to 3 , taking Airbus-321 as an example, the 3D modeling and sectioning of the aerial target are specifically as follows:

[0034] Step S1-1, according to the specific size of the aerial target, the aircraft is proportionally and dimensionally three-dimensionally finely modeled.

[0035] Step S1-2, the modeled aerial target model is triangularly facetted to ensure that the facet size meets the requirements of X-band electromagnetic scattering calculation.

[0036] Step S1-3, electromagnetic simulation calculation is performed by using the 3D model after facet sectioning, and RCS echo data is obtained according to the set imaging parameters. In this embodiment, the simulation frequency band belongs to the X-band. The incident angle is 135°, which means that the ground radar observes from the oblique below of the aircraft target. Table 1 is the ISAR imaging parameters of the aerial target.

[0037] Table 1

[0038]

[0039]

[0040] Step S1-4, imaging is performed by using the calculated RCS echo data to obtain the ISAR image of the aerial target at each observation angle.

[0041] Step S2, the obtained all-around ISAR image is taken as the true value and is divided to obtain a data set with missing samples in the angle dimension, that is, to obtain the training set and the test set.

[0042] Step S2 includes the following sub-steps:

[0043] Figure 4 and Figure 5 are schematic diagrams of the all-around ISAR images of Airbus-321 and Airbus-350 in the embodiment of the present application, respectively.

[0044] As shown in Figure 4 , Figure 5 , the specific steps for obtaining the all-around ISAR image are as follows:

[0045] Step S2-1, after the corresponding air target 3D model is established in step S1, simulation calculation is performed on the models, and imaging simulation is performed every 1°. The ISAR simulation image of each type of target covers 0-360°. The ISAR simulation data set covering the full coverage in the azimuth dimension is taken as the true value, and is used for subsequent comparison of the effect of sample expansion.

[0046] Step S2-2, set the interval degree, in this embodiment, the interval degree is set to 6°, and samples with an interval of 12° are extracted from 0-360° as a training set of the deep learning network model, such as 0°, 12°, 24°, …, 360°, 31 samples are taken as the training set, and the purpose is to generate ISAR images of 0-360° only by using these samples. The remaining ones are taken as the test set. The purpose is to verify that the subsequent deep learning network model can be trained by using the training set with a certain interval between the front and back, and when testing, only the angle is input, and the missing image in the angle can be generated.

[0047] Step S3, build a deep learning network model.

[0048] Figure 6 is a structure diagram of the deep learning network model built in the embodiment of the application.

[0049] As shown in Figure 6 , step S3 can include the following sub-steps:

[0050] Step S3-1, the structure of the deep learning network model used in the embodiment of the application is generally in the form of a conditional generative adversarial network framework, and the generator and the discriminator are two important components of the deep learning network model framework in this embodiment. The generator is used to generate data, and the goal is to generate samples very similar to real data, so that the distribution of generated data constantly approaches the real data. The discriminator is used to distinguish whether the input data belongs to the generated data or the sample data, and the goal is to get a high judgment accuracy. The work of the generator and the discriminator is alternately performed.

[0051] Step S3-2, the generator and the discriminator both adopt the basic structure of: convolution layer-Batch-Norm layer-activation function layer.

[0052] Figure 7 is a generator structure diagram for gradually improving the image resolution in the embodiment of the application.

[0053] As shown in Figure 7 , the working process of the generator of the deep learning network model is specifically:

[0054] Step S3-3, the generator gradually improves the resolution of the generated image, in this embodiment, the first stage trains to generate an image with a resolution of 4*4, and then generates images with resolutions of 8*8, 16*16, …, and 128*128. In this way, the general appearance of the ISAR image can be learned in the previous stage, and the detailed content of the image can be learned in the subsequent steps.

[0055] Step S4, determine the input and output of the deep learning network model, input the data with the type category in the training set into the deep learning network model, and train the generator and the discriminator of the deep learning network model.

[0056] Step S4 includes the following sub-steps:

[0057] As shown in Figure 6 and Figure 7 The specific process of training the generation ability of the deep learning network model is as follows:

[0058] Step S4-1: According to the parameters of the ISAR image to be generated, input random noise z, the category c of the aircraft, and the corresponding angle v of the aircraft into the input end of the generator. For example, for an ISAR image with an azimuth angle of 90°, the angle is converted into cosine and sine values in radian system, which are taken together as an angle vector v=[cosφ, sinΦ] T input into the generator input end. The category is also converted into a one-hot vector composed of 0 and 1. In this way, the angle and the category are converted into vector representations, and they are all regressible variables.

[0059] Step S4-2: The discriminator discriminates whether the generated image is realistic or not, and outputs 1 if it is realistic and 0 if it is not realistic. The discriminator returns such a return value to the generator, so that the image generated by the generator is closer and closer to the image we want to get rather than a random noise image. At the same time, the discriminator also compares the angle and category information with the true value, and trains the generator to generate corresponding images for specific angles and categories.

[0060] Step S4-3: The total loss function of the deep learning network model is shown in formula 1, where v=[cosφ, sinφ] T represents the angle, c represents the category, and z represents the random noise:

[0061]

[0062] Step S5, input the angle and category parameters in the test set, and output the image under the corresponding information.

[0063] Figure 8 is the process diagram of the generator generating the final image based on the deep learning network model built in the embodiment of the application; Figure 9is the generated result graph in the embodiment of the application.

[0064] As shown in Figure 8 and Figure 9 , step S5 can include the following sub-steps:

[0065] Step S5-1, as described in step 2, 31 samples with azimuth angles of 0°, 12°, 24°,..., 360° have been taken out as the training set. As shown in Figure 8 , when testing, the remaining angles that have not been trained, i.e. the parameters in the test set, are used for testing. The angle to be tested and the category are input into the input end of the generator, and the generator outputs the ISAR image corresponding to the angle.

[0066] Step S5-2, compare the generated image of the generator with the true value of the test set reserved in advance. As shown in Figure 9 , compare the generated image of the input angle and category with the true value image. The images on both sides of each row are the images input during training, and the middle is the image generated by only inputting the angle. The comparison between the generated image and the true value. The top row is the true value image, and the bottom row is the generated image.

[0067] The structural similarity SSIM is introduced, and the expression is shown in formula 2. Wherein x, y are two images to be compared, μ x , μ y is the average value of the pixel intensity, σ x , σ y is the standard deviation of the pixel intensity, σ xy is the sample correlation coefficient between the corresponding pixels of x and y, I, C, S represent brightness similarity, contrast similarity and structural similarity respectively. When two images are exactly the same, the SSIM value reaches the maximum value 1. Our result is close to 1.

[0068] SSIM(x, y) = I(x, y) α C(x, y) β S(x, y) γ (2)

[0069] Effects of the embodiment

[0070] According to the deep learning-based multi-azimuth sample expansion method for air target ISAR imaging of the application, in the field of air target ISAR target sample identification, the intelligent target identification technology can obtain sufficient identification ability under the condition of a small number of samples. That is, the intelligent target identification technology still has a high recognition rate under the condition of a small number of samples, and can assist or replace artificial in many application fields. The method solves the problems of difficulty in obtaining air target ISAR images, sample missing, azimuth dimension data missing, and incomplete ISAR images under each angle.

[0071] The above examples are merely intended for illustrating specific embodiments of the present application, and the present application is not limited to the scope described in the above examples.

Claims

1. A deep learning-based multi-aspect sample expansion method for ISAR imaging of an aerial target, used for expanding samples in the aspect dimension of the aerial target, characterized in that, The method comprises the following steps: Step S1, obtaining an all-around aerial target ISAR image simulation data set by modeling the aerial target and calculating the fast electromagnetic scattering; Step S2, dividing the aerial target ISAR image simulation data set by setting interval degrees to establish a training set and a test set; Step S3, building a deep learning network model, which at least has a generator for generating data and a discriminator for constraining the generation result of the generator; Step S4, inputting the data with model category labels in the training set into the deep learning network model to train the generator and the discriminator; Step S5, inputting the data in the test set into the trained deep learning network model to output the image under the corresponding information, The step S1 comprises the following sub-steps: Step S1-1, establishing a proportional and equal-sized aerial target model according to the specific size of the aerial target, Step S1-2, determining the cell size according to the imaging frequency band requirement, and performing triangular cell subdivision on the aerial target model, Step S1-3, performing electromagnetic simulation calculation on the aerial target model after triangular cell subdivision, and obtaining RCS echo data according to the set imaging parameters, Step S1-4, imaging the RCS echo data to obtain the all-around aerial target ISAR image simulation data set.

2. The multi-aspect sample expansion method for aerial target ISAR imaging based on deep learning according to claim 1, wherein: wherein The step S2 extracts the all-around aerial target ISAR image simulation data set by setting interval degrees, takes the extracted aerial target ISAR image simulation data set as the training set, and takes the unextracted as the test set.

3. The multi-aspect sample expansion method for aerial target ISAR imaging based on deep learning according to claim 1, wherein: wherein The deep learning network model in the step S3 is in the form of a conditional generative adversarial network framework, Both the generator and the discriminator comprise a convolution layer, a Batch-Norm layer, and an activation function layer, The generator and the discriminator work alternately.

4. The multi-aspect sample expansion method for aerial target ISAR imaging based on deep learning according to claim 1, wherein: wherein The step S4 comprises the following sub-steps: Step S4-1, inputting training random noise, training category, and training angle in the training set into the input end of the generator to generate an aerial target ISAR image under the corresponding information, Step S4-2, the discriminator discriminates the fidelity of the generated aerial target ISAR image under the corresponding information, and judges whether the expansion data generated by the generator conforms to the training angle and the training category, so as to constrain the generation result of the generator, Step S4-3, calculating the total loss function of the deep learning network model, and the calculation formula is as follows: where v = [cosφ, sinφ] T for the training angle, c is the training class, and z is the training random noise.

5. The multi-aspect sample expansion method for aerial target ISAR imaging based on deep learning according to claim 1, wherein: wherein, The step S5 comprises the following sub-steps: Step S5-1, inputting the test angle and test category in the test set into the input end of the trained generator to obtain the ISAR image of the aerial target under the corresponding information, Step S5-2, comparing the generated ISAR image of the aerial target under the corresponding information with the test set in step S2 to test the generation effect.

Images