Radar target open set identification method based on CVAE-GAN depth model

Abstract

The invention discloses a radar target open set identification method based on a CVAE-GAN depth model, and the method comprises the steps: obtaining an embedded target data set, and deducing a CVAE model through the attributes of an embedded target; the classification loss and the GAN loss are integrated into a CVAE model, and a CVAE-GAN model is obtained; the embedded target data set is divided into a training set and a test set, and training and testing of the CVAE-GAN model are realized through the training set and the test set; and inputting a radar target image to be identified into the CVAE-GAN model to realize radar target open set identification. According to the method, an encoder network and a generative network in the CVAE-GAN are combined, and a missing clear radar target image in a training data set can be generated under the condition that a target category label and a target attribute are given; and meanwhile, known categories can be classified, and unknown target categories of which the features are not in a known category angle range can be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of radar target open set recognition, and in particular to a radar target open set recognition method based on a CVAE-GAN deep model. Background Art

[0002] Target recognition based on radar images includes two stages: first, extracting the recognition features of radar target images, and then using a classifier to assign the image to a specific category based on the recognition features. Recognition features are the key to the classifier and have an important impact on the success of classification. In recent decades, a large number of feature extraction techniques have been proposed in the field of target recognition based on radar images, including methods based on principal component analysis (PCA), methods based on scattering center model (SCM), sparse representation methods, and multi-feature fusion methods. Unlike traditional techniques for manually extracting image features, deep learning (DL) technology automatically extracts image features through the combination of feature extractors and classifiers, thereby achieving better target recognition performance. At present, on some public SAR target recognition datasets, DL technology has achieved recognition accuracy far exceeding that of traditional technologies. However, there are obvious limitations in the current research on radar target recognition based on DL. On the one hand, many existing DL models are "black boxes" and their predictions cannot be explained in a way that humans can understand, so it is difficult to ensure the rationality and reliability of the prediction model. On the other hand, most of the existing target recognition methods assume that the target classes in the test set have already appeared in the training set. However, in real scenarios, there may be some unknown target classes. For unknown classes, traditional DL object recognition methods do not work well.

[0003] In order to address the above challenges, some studies have been conducted. Transfer learning was studied to transfer knowledge from some SAR datasets with rich labeled samples to target tasks with limited labeled samples. The recognition model was first trained on simulated SAR data, and then the recognition model was transferred to the target task. The simulated SAR images were processed to reduce the difference between the simulated images and the real images. In addition to transfer learning methods, generative networks are also used to generate SAR images that are not included in the training data. The above studies help improve classification results when there are few samples of SAR target classes. However, for target classes with no samples, using the FSL method is ineffective. To address this problem, several zero-shot learning methods were studied in SAR-ATR. In the literature "T. Toizumi, K. Sagi, and Y. Senda, "Automatic association between SAR and optical images based on zero-shot learning," in Proc. IEEE Int. Geosci. Remote Sens. Symp. IGARSS, Jul. 2018, pp. 17–20. " and "Q. Song, H. Chen, F. Xu, and TJ Cui, "EM simulation-aided zero-shot learning for SAR automatic target recognition," IEEE Geosci. Remote Sens. Lett., vol. 17, no. 6, pp. 1092–1096, Jun. 2020. ", optical images or simulated SAR images of unknown class targets are used to provide auxiliary information for zero-shot target recognition. Without adding additional information, a feature space of SAR-ATR is constructed based on known classes. Mapping SAR images of unknown class targets into the feature space can explain the properties of the target to a certain extent. In addition, the feature space is continuous and can generate images of targets that are not in the training data, although the generated images are usually blurred. A more stable feature space is established with better interpretability. The problem is that the feature space can only qualitatively explain the unknown classes, but cannot effectively distinguish the unknown classes from the known classes. Open set recognition (OSR) is a task similar to zero-shot recognition. In addition to a few known classes, OSR treats all unknown classes as an additional class. In testing, it can classify targets of known classes and reject targets of unknown classes. Some research has been carried out on open set recognition of SAR targets, such as methods based on support vector machines (SVM) and marginal sample selection methods. These methods do not fully utilize the powerful feature extraction capabilities of DL.Recently, a SAR-ATR algorithm based on Generative Adversarial Network (GAN) was proposed. A classification branch was added to the discriminator, which enables the discriminator to provide target categories and probability scores of the target belonging to known categories. However, the recognition accuracy of this method decreases significantly as the number of known classes increases. In addition, like existing DL models, this model also lacks transparency and interpretability. Summary of the invention

[0004] The purpose of this invention is to provide a radar target open set recognition method based on the CVAE-GAN deep model, which combines the encoder network and the generator network in CVAE-GAN to generate clear radar target images missing in the training data set given the target category label and target attributes.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A radar target open set recognition method based on CVAE-GAN deep model includes the following steps:

[0007] S1. Obtain the embedded target dataset and derive the CVAE model through the attributes of the embedded target;

[0008] S2. Integrate the classification loss and GAN loss into the CVAE model to obtain the CVAE-GAN model;

[0009] S3, dividing the embedded target data set into a training set and a test set, training and testing the CVAE-GAN model through the training set and the test set, and obtaining a trained CVAE-GAN model;

[0010] S4. Input the radar target image to be identified into the trained CVAE-GAN model to realize open set recognition of radar targets.

[0011] Preferably, the derivation process of deriving the CVAE model by embedding the properties of the target in S1 is as follows:

[0012] The generation of the CVAE model is achieved by maximizing the conditional log-likelihood function:

[0013] max E x～p(x) log p(x|y,θ) (6.1)

[0014] Among them, p(x), p(x|y,θ) are the prior distribution and posterior distribution of x. According to the idea of ​​VAE, an auxiliary latent variable z is added to represent logp(x|y,θ); let q(z|x) be the posterior distribution of z, then logp(x|y,θ) is expressed as:

[0015]

[0016] Among them, KL[q(z|x)||q(z|y,θ)] is the KL divergence distributed between q(z|x) and q(z|y,θ); the variational lower bound of KL[q(z|x)||q(z|y,θ)] is obtained as:

[0017]

[0018] Therefore, maximizing logp(x|y,θ) is transformed into maximizing its variational lower bound, which is derived as follows:

[0019]

[0020] Considering that the latent variable z is generated by y, θ through p(z|y, θ), the term logp(x|y, θ, z) in equation (6.4) is simplified to logp(x|z); therefore, the model generation problem is reformulated as:

[0021]

[0022] Preferably, in S2, the classification loss is integrated into the CVAE model, specifically including:

[0023] In formula (6.5), the expectation symbol E is omitted. x～p(x) ; Increase classification loss Ε z～q(z|x) logp(y|z), the improved CVAE model is expressed as follows:

[0024]

[0025] Where x, y, θ represent the parameters of the target image, class label and related attributes respectively, z is an auxiliary latent variable, q(z|x) is the posterior distribution of z when x is known, p(z|y,θ) is the posterior distribution of z when y and θ are known, p(x|z) is the posterior distribution of x when z is known, p(y|z) is the posterior distribution of y when z is known, -KL[q(z|x)||p(z|y,θ)] is the distribution loss, and Ε z～q(z|x) logp(x|z) is the reconstruction loss, Ε z～q(z|x) logp(y|z) is the classification loss.

[0026] Preferably, in S2, the GAN loss is integrated into the CVAE model, and the resulting CVAE-GAN model is as follows:

[0027]

[0028] Among them, E x～p(x|z)[logD(x)] is the GAN loss, D(·) is the additional discriminator;

[0029] The adversarial loss of the additional discriminator is expressed as:

[0030]

[0031] Preferably, the distribution loss formula of the CVAE-GAN model shown in formula (6.7) is expressed as follows:

[0032]

[0033] Among them, q(z|x) and p(z|y,θ) obey the normal distribution q(z|x)=N(μ1(x),σ1(x)) and p(z|y,θ)=N(μ2(y,θ),σ2(y,θ)) respectively; the mean μ1(x) and standard deviation σ1(x) are functions of x, and μ2(y,θ) and σ2(y,θ) are functions of y and θ. μ1(x), σ1(x), μ2(y,θ) and σ2(y,θ) are expressed as μ1, σ1, μ2, σ2.

[0034] Preferably, fix σ2 and set σ2=1, then equation (6.9) is modified to

[0035]

[0036] Since the latent variables are multi-dimensional, the final loss is the sum of all dimensions of L1.

[0037] Preferably, the reconstruction loss formula of the CVAE-GAN model shown in formula (6.7) is expressed as follows:

[0038]

[0039] Among them, q(x|z) obeys the normal distribution N(μ3(z),σ z ), where μ3(z) is a function of z, σ z is a constant.

[0040] Preferably, the classification loss of the CVAE-GAN model shown in formula (6.7) is specifically the cross entropy loss, and the feature corresponding to the i-th target sample is z i , the class label is y i , the cross entropy loss is expressed as:

[0041]

[0042] Among them, f j =s cos(φ j ), φ j and They are z i and W j , The angle between them, m is the added angular margin.

[0043] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned radar target open set recognition methods based on a CVAE-GAN deep model.

[0044] According to the specific embodiment provided by the present invention, the present invention discloses the following technical effects: the CVAE-GAN model proposed by the present invention can obtain the feature space of the radar target image, which has three main advantages:

[0045] (1) Conditional generation of target images: Combining the encoder network and the generator network in CVAE-GAN, given the target category label and target attributes, clear radar target images missing in the training dataset can be generated. This method can be used for data enhancement and is beneficial for radar target recognition under limited labeled data conditions.

[0046] (2) Features have certain interpretability: Unlike autoencoders, the latent variables of VAE are continuous in the variable space. Therefore, the latent variables obtained by the encoder network can be used as a representation of the SAR image, reflecting certain attributes of the target.

[0047] (3) Identification of unknown classes: By adding the classification angle margin in the feature space, the intra-class distance of the known classes is reduced and the inter-class distance is increased. In this way, the known classes can be classified and the unknown target classes whose features are not within the angle range of the known classes can be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flow chart of a radar target open set recognition method based on a CVAE-GAN deep model provided by the present invention;

[0050] Figure 2 Schematic diagram of angular margin for binary classification;

[0051] Figure 3a This is the overall architecture diagram of the CVAE-GAN network;

[0052] Figure 3bThis is the specific structure diagram of each subnet of the CVAE-GAN network;

[0053] Figure 4 This is the visualization result diagram of 7 types of known target image features at a pitch angle of 15°;

[0054] Figure 5 for Figure 4 Azimuth marking result diagram of each sample shown;

[0055] Among them, (a) is the global view; (b) is the local enlarged view;

[0056] Figure 6a The visualization result diagram of 7 types of known target image features at pitch angles of 15° and 17°;

[0057] Figure 6b This is the image feature visualization result diagram of 7 types of known targets and 3 types of unknown targets at a pitch angle of 15°;

[0058] Figure 7 Generate partial result display graph for the condition;

[0059] Figure 8 Feature visualization for conditional generated images;

[0060] Fig. 9 The missing angle images in the generated training set;

[0061] Fig.10 for Fig. 9 Generate feature visualizations of images;

[0062] Fig.11 Visualize the classifier class weight vector;

[0063] Fig.12 Generate results for images based on the classifier class weight vector;

[0064] Fig.13 It is a confusion matrix diagram. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] like Figure 1 As shown, a radar target open set recognition method based on the CVAE-GAN deep model includes the following steps:

[0068] S1. Obtain the embedded target dataset and derive the CVAE model through the attributes of the embedded target;

[0069] S2. Integrate the classification loss and GAN loss into the CVAE model to obtain the CVAE-GAN model;

[0070] S3, dividing the embedded target data set into a training set and a test set, training and testing the CVAE-GAN model through the training set and the test set, and obtaining a trained CVAE-GAN model;

[0071] S4. Input the radar target image to be identified into the trained CVAE-GAN model to realize open set recognition of radar targets.

[0072] In a specific embodiment, the method of the present invention specifically comprises:

[0073] 1.1CVAE-GAN Model

[0074] We hope to generate the corresponding target image given the class label and attribute parameters such as observation angle, resolution, frequency and polarization mode. Let x, y, θ represent the parameters of the target image, class label and related attributes respectively. In the simulation example, θ specifically refers to the target angle. The generative model can be implemented by maximizing the conditional log-likelihood function

[0075] max E x～p(x) logp(x|y,θ) (6.1)

[0076] Among them, p(x), p(x|y,θ) are the prior distribution and posterior distribution of x. According to the idea of ​​VAE, an auxiliary latent variable z is added to represent logp(x|y,θ). Let q(z|x) be the posterior distribution of z, then logp(x|y,θ) can be expressed as

[0077]

[0078] where KL[q(z|x)||q(z|y,θ)] is the KL divergence between the distributions q(z|x) and q(z|y,θ). Due to the non-negativity of the KL divergence, we can get its variational lower bound as

[0079]

[0080] Therefore, maximizing logp(x|y,θ) can be transformed into maximizing its variational lower bound. The variational lower bound can be further derived

[0081]

[0082] Considering that the latent variable z can be generated from y and θ through p(z|y,θ), the term logp(x|y,θ,z) in equation (6.4) is simplified to logp(x|z). That is, the latent variable z contains the information of the target class label and target attribute, which can be used to generate the corresponding target image. Therefore, the generation problem in equation (6.1) can be reformulated as

[0083]

[0084] In formula (6.5), for simplicity, the expected symbol E is omitted. x～p(x) . The first term in equation (6.5) aims to minimize the difference between the distribution q(z|x) and p(z|y,θ). Therefore, the latent variable z generated by x will have the information of y,θ. This makes conditional image generation and subsequent target classification possible. The purpose of the second term in equation (6.5) is to maximize the probability of generating the corresponding image x from z, where z is obtained by q(z|x). The present invention refers to the above two losses as distribution loss and reconstruction loss respectively.

[0085] Since the latent variable z contains the target category information, it can be used as an identification feature for target classification. The classification loss Ε is added to equation (6.5) z～q(z|x) logp(y|z), the problem can be further expressed as

[0086]

[0087] Considering that the images generated by VAE are often blurry, the present invention combines GAN with CVAE to improve image quality. Let D(·) be an additional discriminator, then the problem of the generative model at this time can be expressed as

[0088]

[0089] The adversarial loss of the discriminator is expressed as

[0090]

[0091] Equation (6.7) and Equation (6.8) are the final objective functions of the CVAE-GAN model, and the two are solved alternately during training.

[0092] 1.2 Implementation of loss function

[0093] The specific form of the loss function in formula (6.7) is derived as follows:

[0094] ①Distribution loss

[0095] Assume that q(z|x) and p(z|y,θ) in the first term of equation (6.7) obey the normal distribution q(z|x)=N(μ1(x),σ1(x)) and p(z|y,θ)=N(μ2(y,θ),σ2(y,θ)) respectively. The mean μ1(x) and standard deviation σ1(x) are functions of x, and μ2(y,θ) and σ2(y,θ) are functions of y and θ. Unlike the original VAE, which uses a standard normal distribution for all samples, the distribution of different samples here can be different according to the target attributes. Therefore, samples with different target attributes can be distinguished in the feature space. To simplify the representation, μ1(x), σ1(x), μ2(y,θ), and σ2(y,θ) are denoted by μ1, σ1, μ2, and σ2 respectively, then the distribution loss in equation (6.7) can be expressed as

[0096]

[0097] If both σ1 and σ2 are variable, in order to maximize L1, the possible results of σ1 and σ2 will be close to 0. In this case, the desired randomness in VAE will disappear. To avoid this problem, in this invention, we fix σ2 and set σ2 = 1. Then equation (6.9) can be modified to

[0098]

[0099] Since the latent variables are multi-dimensional, the final loss is the sum of all dimensions of L1.

[0100] ②Reconstruction loss

[0101] Assume that q(x|z) in the second loss in equation (6.7) follows a normal distribution N(μ3(z),σ z ), where μ3(z) is a function of z, σ z is a constant. Therefore, the reconstruction loss is expressed as

[0102]

[0103] From formula (6.11), we can see that maximizing L2 is equivalent to minimizing ||x-μ3(z)|| 2 .

[0104] ③Classification loss

[0105] Cross entropy loss (CEL) is a commonly used classification loss. For the i-th target sample, its corresponding feature is z i , the class label is y i , then its cross entropy loss (maximization) can be expressed as

[0106]

[0107] Among them, C is the number of known target classes, For z i The score belonging to the jth class, W j ,b j are the weights and biases corresponding to the jth class in the classifier. However, as mentioned in the introduction, in practice there may be some unknown classes that traditional CEL cannot distinguish from known classes. In order to identify unknown classes, one solution is to add a classification margin to the softmax loss function, thereby forcing the classification boundary to be closer to the class center vector, thereby increasing the separability between classes. This has been a hot research direction in face recognition in recent years. In this invention, we apply ArcFace to add the angular margin of known classes in the feature space. Let |W j |=1,b j =0,|z i |=s, s is a predetermined constant. Then the new classification loss can be expressed as

[0108]

[0109] In the formula, f j =scos(φ j ), φ j and They are z i and W j , The angle between them, m is the added angular margin. In formula (6.13), different classes can be separated by angle. For the simplest binary classification, the classification boundaries of class 1 and class 2 are scos(φ1+m)=scos(φ2) and scos(φ1)=scos(φ2+m), respectively. The principle diagram is shown in Figure 2 When m = 0, the two classification boundaries overlap and the unknown class cannot be identified; when m > 0, the two classification boundaries are separated. If the unknown class is between the two classification boundaries, the unknown class can be distinguished from the known class.

[0110] The fourth term in equation (6.7) is implemented using the traditional GAN ​​loss.

[0111] 1.3 Network Structure

[0112] Based on the above model, the overall network of CVAE-GAN proposed in this invention is as follows Figure 3aAs shown. It consists of 5 sub-networks, namely encoding networks E1 and E2, generation network G, classification network C and identification network D. E1, E2, G, and C are used to fit q(z|x), p(z|y,θ), p(x|z) and p(y|z) in equation (6.7) respectively. More specifically, E1 is used to implement the functions μ1(x), σ1(x) in L1, E2 is used to implement the function μ2(y,θ) in L1, and G is used to implement the function μ3(z) in L2. The encoder network and the generation network are used together to realize the conditional generation function. Using E1 to project the image into the feature space and using G to generate the corresponding target image based on the features can explain the properties of the target to a certain extent. In addition, E1 and C can be used to classify known classes and identify unknown classes.

[0113] The specific structure of the sub-network used in the experimental part is as follows Figure 3b As shown. In the figure, Conv is a convolutional layer, and ConvT is a transposed convolutional layer. Conv(c_in, c_out, k, s, p) means that the convolutional layer has c_in input channels, c_out output channels, a convolution kernel size of k×k, parameters stride=s, padding=p. Linear(a_in, a_out) means a fully connected layer with input dimension a_in and output dimension a_out. BN and DP are abbreviations for BatchNorm and Dropout. The classifier C is constructed according to formula (6.13). Figure 3b The total number of parameters of CVAE-GAN is about 33.46 million.

[0114] In network training, the reparameterization technique is used to sample z, the purpose of which is to make the sampling process differentiable. The sampling process can be expressed as

[0115] z=μ1+σ1×ε(6.14)

[0116] Where ε is the noise ε:N(0,1) that obeys the standard normal distribution, μ1=μ(x),σ1=σ(x). Therefore, the sampling result z satisfies the normal distribution q(z|x)=N(μ(x),σ(x)). The second term in equation (6.7) can be expressed as the sum of the samples of z

[0117]

[0118] Since the network is usually trained in small batches, the number of samples of z in equation (6.15) can be set to K = 1 in each iteration. Similarly, for E in equation (6.7) z～q(zx) [logp(yz)] and E x～p ( xz) [logD(x)], z is sampled only once per training.

[0119] 2 Experimental analysis

[0120] This section will use the commonly used public MSTAR dataset to test the proposed feature learning method. There are 10 types of vehicle targets in the dataset, including 2S1 (self-propelled howitzer), BDRM2 (armored reconnaissance), BTR60 (armored personnel carrier), D7 (bulldozer), T62 (tank), ZIL131 (military truck), ZSU234 (air defense force), T72 (main battle tank), BMP2 (infantry fighting vehicle) and BTR70 (armored personnel carrier). They are numbered from Class0 to Class9 in sequence. The first 7 classes are selected as known target classes, and the last 3 classes are selected as unknown classes. The 7 classes of known target images with a pitch angle of 15° are used for training, and the 10 classes of target images (including 7 known classes and 3 unknown classes) with a pitch angle of 17° are used for testing. During training, the input of the network includes SAR images, azimuth angles, and corresponding class labels. All SAR images are cropped to 128×128 size. The azimuth information is represented by a two-dimensional vector [cosφ, sinφ] T , where φ∈[0°,360°) represents the azimuth. The class label is encoded as a 7-dimensional one-hot vector. The dimension of the latent variable is set to 512. In the experiment, the classifier constants are set to s=64,m=1.

[0121] In order to make the training of CVAE-GAN easier, a step-by-step training strategy is adopted. First, the loss (L1+L2) is used to train E1, E2, and G in the CVAE network. Then, the classification network C and the CVAE network are jointly trained using the loss (L1+L2+L3). Finally, the losses L1+L2+L3+L4 and L D Adversarial training will be performed to improve the clarity of the generated images. The network is initialized with default parameters in Pytorch and trained using Adam optimization with a learning rate of 0.0001. All experiments are performed on a workstation configured with an Intel Core i9-9900 CPU and an NVIDIA GeForce RTX 2080Ti GPU.

[0122] 2.1 Feature Space Visualization

[0123] The latent variable (i.e., the extracted feature) is a high-dimensional vector of 512 dimensions, which is difficult to visualize directly. Therefore, t-SNE, a widely used dimensionality reduction technique, is used to embed the high-dimensional features into a two-dimensional feature space. Since only the angle of the feature is used for classification in formula (6.13), all features are normalized before visualization. Figure 4The figure shows the visualization of the extracted features of 7 classes of known target images with a pitch angle of 15°. The colors of Class 0 to Class 6 are "blue", "orange", "green", "red", "purple", "brown" and "pink", respectively. It can be seen that in the feature space, different classes can be well distinguished. Figure 4 The azimuth angles of the features in Figure 5 As shown in (a)-(b) in the figure, the results show that with the continuous change of azimuth angle, the SAR image features also change continuously in space. Therefore, the extracted features are helpful to explain the angular properties of SAR images.

[0124] In addition, the features of 7 known classes in the test data with a pitch angle of 17° are also examined. All these extracted features are marked in gray, such as Figure 6a As shown in Figure 2, it can be seen that the characteristics of the test data are generally similar to the distribution of the characteristics of the training data. Figure 6b Features of three unknown classes and known classes are shown, where the unknown classes Class 7, Class 8, and Class 9 are marked with "olive", "cyan", and "grey", respectively. It can be seen that in the feature space, most samples of the unknown classes are separated from samples of the known classes, which makes it possible to distinguish the unknown from the known. It can also be seen that Class 7 (T72) is close to Class 4 (T62) in the feature space. This indicates that there may be a high similarity between the two classes. This is consistent with the physical fact that the T72 tank is close to the T62 tank in shape and structure. Similar conclusions can be drawn for Class 9 (BTR70) and Class 2 (BTR60). Therefore, for SAR images of unknown categories, the feature space helps to interpret the target attributes.

[0125] 2.2 Image Generation Results

[0126] In this section, the image generation ability of the proposed model is tested. Figure 7 The images generated under given conditions are shown. The first row is 8 original SAR images randomly selected from the training data set. The second row is the result of the original SAR image x after encoding z=μ1(x) and decoding μ3(z). Take the class label and azimuth of the original SAR image, encode z=μ2(y,θ) and decode μ3(z), and get the conditionally generated image, as shown in the third row of the figure. It can be seen that both the encoded and decoded images and the conditionally generated images are very similar to the original SAR image. Conditional generation also considers additive noise. The feature after adding noise can be expressed as z=μ2(y,θ)+ε, where ε is in the range [-U ε ,U ε ] is uniformly distributed noise. The image generated after adding noise is as follows Figure 7As shown in rows 4 to 8 of . We can see that although there are some slight differences in the randomly generated images, they are still similar to the original images. Figure 8 The normalized features of the noisy image are shown (marked in grey), demonstrating the continuous generative capability of the feature space.

[0127] Next, we will generate images that are not in the training data (images are missing at certain azimuth angles). In the training data set, Class 0 to 6 have obvious missing samples in the azimuth angle ranges of (83.3, 93.3), (275.5, 289.5), (200.3, 210.3), (84.5, 94.5), (83.1, 93.1) and (83.0, 93.0), respectively. Perform uniform sampling at equal intervals in the missing angle range to generate SAR images for the sampling angles and class labels. The results are shown in the figure. Fig. 9 As shown in the figure, x(θ1) and x(θ2) represent the original SAR images with azimuth angles of θ1 and θ2 respectively, μ3(μ2(y,θ1)) and μ3(μ2(y,θ2)) represent the conditionally generated SAR images with azimuth angles of θ1 and θ2 respectively, μ3(μ2(y,θ′)) represents the SAR image that does not exist in the training set based on the missing angle θ′, and θ1′, θ2′, θ3′, θ4′ are equally spaced sampling values ​​in the interval of θ1 and θ2. The first and last columns in the figure are the original SAR images with angles at the edge of the missing interval, and the middle column is the generated image of the sampling angle. The visual features of the generated image (marked in gray) are shown as follows: Fig.10 As shown in Figure 2, it can be seen that the generated image changes smoothly and can well fill the missing angles in the feature space.

[0128] In addition, the class weight vectors of each class of the classifier are visualized in the feature space. The results are as follows: Fig.11 As shown in the figure, the weight vector is marked with a gray '9'. We can see that the weight vector is very close to the features of many samples of the corresponding class. In the classifier, images with the same features as the weight vector will get the highest classification score of the corresponding class. Next, we will use the weight vector as a feature to show the generated image. The feature amplitude will affect the result of the generated image. The mean amplitude of the original weight vector is about 198, and the mean feature amplitude in the training data is about 70. Here, we set weight vectors with different amplitudes. The generated image is as follows Fig.12 As shown, rows 1 to 7 correspond to different amplitudes, and columns 1 to 7 correspond to Class 0 to 6. The results can reflect the most recognizable image features of each class. For example, Fig.12The images generated in the third column all have a strong clutter background. This reflects that in some SAR images of Class 2, the clutter is strong, while in SAR images of other classes, the strong clutter features are not obvious. Therefore, for SAR images with strong clutter background, the classifier will classify them as Class 2 with a high probability. By sampling the feature space and generating images, the decision behavior of the classifier can be explained to a certain extent.

[0129] 2.3 Unknown Category SAR Target Recognition Results

[0130] In this section, we will test the proposed model's ability to classify unknown classes. SAR images with a depression angle of 17° are used for testing. There are 7 known classes (2049 samples) and 3 unknown classes (698 samples) in the test dataset. Figure 2 As shown in the figure, the target within the classification boundary will be classified into the corresponding known class (class 0 to class 6), while the target outside the classification boundary of the known class will be classified into the unknown class. The ratio of correctly classified samples in the known class is recorded as the true positive rate TPR, and the ratio of incorrectly classified samples in the unknown class is recorded as the false positive rate FPR. In the test, by using the classification threshold m th To adjust the classification boundary. The recognition results under different classification thresholds are shown in Table 1. It can be found that as m th As the value of m increases, both TPR and FPR decrease. th When =0.7, the overall accuracy reaches the best result of 0.9236.

[0131] Table 1

[0132]

[0133] To further illustrate the results, Fig.13 m is shown in th =0.7. It can be seen that there are no misclassified samples in the 7 known classes, and there are only some misclassified samples between the known classes and the unknown classes. According to the confusion matrix, the average values ​​of precision, recall and F1_score of the 8 classes can be calculated by formula (6.16), where is the total number of classes. th When =0.7, the precision is 0.9410, the recall is 0.9359, and the F1_score is 0.9369.

[0134]

[0135] In the above experiments, the classification angle margin is m = 1. We also tested the recognition results under different classification angle margins, and the results are shown in Table 2. It can be seen that all the accuracy, precision, recall and F1_score increase with the increase of m. When m = 1.4, m th = 0.9, the overall classification accuracy reaches 0.9363. It should be noted that when m is too large, the network will be difficult to converge.

[0136] Table 2

[0137]

[0138] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for implementing any of the above-mentioned radar target open set recognition methods based on a CVAE-GAN deep model.

[0139] The present invention studies a radar target open set recognition method, which can classify known classes, eliminate unknown classes, and has certain interpretability. The present invention obtains a feature space that can well represent the attributes of SAR targets and has the ability to generate images. Variational Auto-encoder (VAE) and GAN are the two most popular deep generation models, both of which can generate images by inputting random noise. The noise in VAE satisfies a certain distribution and can be regarded as a feature representation of the image. VAE is strict in theory, but the generated images are often blurred. GAN has no clear assumptions about the noise distribution and can usually produce clear images. However, the input noise of GAN cannot be used as a representation of the image, and the interpretability is not enough. In addition, traditional VAE and GAN cannot perform conditional generation. Existing studies have shown that combining VAE and GAN can improve the quality of generated images. Therefore, this paper proposes a SAR-ATR feature learning method (denoted as CVAE-GAN) combining conditional VAE (conditionalVAE, CVAE) and GAN for radar target open set recognition. The noise latent variable space in VAE is used as the feature space, in which the target is represented and classified. The attributes and class labels of the target are used as conditions to control image generation, and an additional discriminator is used to improve the quality of the generated image.

[0140] The main goal of the present invention is to find an interpretable feature space that can well characterize the target attributes of radar images and distinguish unknown categories. Generative models are a good choice for achieving interpretability. VAE is selected as a generative model because it not only has the ability to generate, but also can obtain the feature representation of the image. Considering that only random images can be generated in the original VAE, in the present invention, we will first derive a conditional VAE model by embedding target attributes. Then the classification loss and GAN loss are integrated into the CVAE model to obtain the CVAE-GAN model. This model has the ability of conditional image generation, feature interpretability and target classification at the same time. Finally, a deep network will be built based on this model.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0142] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A radar target open set recognition method based on CVAE-GAN deep model, characterized in that: The following steps are involved: S1. Obtain the embedded target dataset and derive the CVAE model through the attributes of the embedded target; S2. Integrate the classification loss and GAN loss into the CVAE model to obtain the CVAE-GAN model; S3, dividing the embedding target data set into a training set and a test set, training and testing the CVAE-GAN model through the training set and the test set, and obtaining a trained CVAE-GAN model; S4. Input the radar target image to be identified into the trained CVAE-GAN model to realize open set recognition of radar targets.

2. According to claim 1, a radar target open set recognition method based on CVAE-GAN deep model is characterized in that: The derivation process of deriving the CVAE model by embedding the attributes of the target in S1 is as follows: The generation of the CVAE model is achieved by maximizing the conditional log-likelihood function: ma xE x～p(x) logp(x|y,θ) (6.1) Among them, p(x), p(x|y,θ) are the prior distribution and posterior distribution of x. According to the idea of ​​VAE, an auxiliary latent variable z is added to represent log p(x|y,θ); let q(z|x) be the posterior distribution of z, then log p(x|y,θ) is expressed as: Among them, KL[q(z|x)||q(z|y,θ)] is the KL divergence distributed between q(z|x) and q(z|y,θ); the variational lower bound of KL[q(z|x)||q(z|y,θ)]0 is obtained as: Therefore, maximizing log p(x|y,θ) is transformed into maximizing its variational lower bound, which is derived as: Considering that the latent variable z is generated by y, θ through p(z|y, θ), the term log p(x|y, θ, z) in equation (6.4) is simplified to logp(x|z); therefore, the model generation problem is reformulated as:

3. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 2 is characterized in that: In S2, the classification loss is integrated into the CVAE model, specifically including: In formula (6.5), the expectation symbol E is omitted. x～p(x) ; Increase classification loss Ε z～q(zx) logp(yz), the improved CVAE model is expressed as follows: Where x, y, θ represent the parameters of the target image, class label and related attributes respectively, z is an auxiliary latent variable, q(z|x) is the posterior distribution of z when x is known, p(z|y,θ) is the posterior distribution of z when y and θ are known, p(x|z) is the posterior distribution of x when z is known, p(y|z) is the posterior distribution of y when z is known, -KL[q(z|x)||p(z|y,θ)] is the distribution loss, and Ε z～q(z|x) log p(x|z) is the reconstruction loss, Ε z～q(z|x) log p(y|z) is the classification loss.

4. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 3 is characterized in that: In S2, the GAN loss is integrated into the CVAE model to obtain the CVAE-GAN model, and the formula is as follows: Among them, E x～p(x|z) [log D(x)] is the GAN loss, D(·) is the additional discriminator; The adversarial loss of the additional discriminator is expressed as:

5. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 4 is characterized in that: The distribution loss formula of the CVAE-GAN model shown in formula (6.7) is expressed as follows: Among them, p(z|x) and p(z|y,θ) obey the normal distribution q(z|x)=N(μ1(x),σ1(x)) and p(z|y,θ)=N(μ2(y,θ),σ2(y,θ)) respectively; the mean μ1(x) and standard deviation σ1(x) are functions of x, and μ2(y,θ) and σ2(y,θ) are functions of y and θ. μ1(x), σ1(x), μ2(y,θ) and σ2(y,θ) are expressed as μ1, σ1, μ2, σ2.

6. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 5 is characterized in that: Fix σ2 and set σ2 = 1, then equation (6.9) is modified to Since the latent variables are multi-dimensional, the final loss is the sum of all dimensions of L1.

7. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 6 is characterized in that: The reconstruction loss formula of the CVAE-GAN model shown in formula (6.7) is expressed as follows: Among them, q(x|z) obeys the normal distribution N(μ3(z),σ z ), where μ3(z) is a function of z, σ z is a constant.

8. The radar target open set recognition method based on the CVAE-GAN deep model according to claim 7 is characterized in that: The classification loss of the CVAE-GAN model shown in formula (6.7) is specifically the cross entropy loss. The feature corresponding to the i-th target sample is z i , the class label is y i , the cross entropy loss is expressed as: Among them, f j =s cos(φ j ), φ j and They are z i and W j , The angle between them, m is the added angular margin.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an open set recognition method for radar targets based on a CVAE-GAN deep model is implemented according to any one of claims 1 to 8.

Images