A Radar Target Open Set Recognition Method Based on Hierarchical Reconstruction Latent Representation Learning

Abstract

This invention discloses an open-set radar target recognition method based on hierarchical reconstruction latent representation learning, applied to the field of radar signal processing. Almost all existing radar open-set recognition methods typically employ fully supervised learning to extract representative features, which only emphasizes the non-discriminatory features of known classes and is unsuitable for representing features of unknown classes. This invention first proposes an unsupervised representation learning method based on a hierarchical fusion reconstruction network (HFRNet) to compensate for the information lost in supervised representation and obtain preliminary closed-set results. Then, Openmax is used to correct the closed-set recognition score and provide the probability of unknown classes, achieving effective open-set recognition of SAR images. Finally, experimental results based on a real-world dataset demonstrate the superiority of this proposed method.

Description

Technical Field

[0001] This invention belongs to the field of radar signal processing, and specifically relates to a radar target open set recognition technology. Background Technology

[0002] Synthetic Aperture Radar (SAR), as an important and powerful modern microwave sensor system in the field of remote sensing, has always played a vital role in target information acquisition in both military and civilian applications. In particular, SAR Automatic Target Recognition (ATR) has attracted widespread attention, and related innovative research continues to emerge.

[0003] In recent years, deep learning-based SAR ATR methods have achieved significant performance in closed-set recognition tasks. The paper "Changjie Cao, Zongjie Cao, and Zongyong Cui, Ldgan: A synthetic apertureradar image generation method for automatic target recognition IEEE Transactions on Geoscience and Remote Sensing, vol.58, no.5, pp.3495-3508, 2020" proposes a novel image-to-image generation method that provides labeled samples for training the recognition model, effectively supplementing ATR with target information when training samples are insufficient. However, it does not consider the possibility of unknown classes appearing in the training process in the real world. Existing research on open set recognition in radar typically favors Openmax as the classifier to correctly identify known classes and reject unknown classes. The paper "Elisa Giusti, Selenia Ghio, Amir Hossein Oveis, and Marco Martorella, Open set recognition in synthetic aperture radar using the openmax classifier, in 2022 IEEE Radar Conference (RadarConf22), 2022, pp. 1-6." first introduced the Openmax classifier to solve the open set recognition problem in SAR images, and experimental results showed that it outperformed the traditional softmax classifier. Then, the paper "Amir Hossein Oveis, Elisa Giusti, Selenia Ghio, and Marco Martorella, Extended openmax approach for the classification of radar images with a rejection option, IEEE Transactions on Aerospace and Electronic Systems, vol. 59, no. 1, pp. 196-208, 2023." extended the Openmax method and proposed a more accurate and robust SAR open set recognition classifier without prior knowledge of the unknown incoming data. However, almost all existing radar open set recognition methods typically employ fully supervised learning to extract representative features, which only emphasizes the non-discriminative features of known classes and is not applicable to feature representation of unknown classes. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a radar target open set recognition method based on hierarchical reconstruction latent representation learning. This method utilizes an Openmax classifier, unsupervised representation learning, and a hierarchical fusion reconstruction network (HFRNet) to achieve known category recognition and unknown category detection for SAR images.

[0005] The technical solution adopted in this invention is: a radar target open set recognition method based on hierarchical reconstruction latent representation learning, comprising:

[0006] S1. Use SAR images from closed-set recognition tasks of known classes to train the hierarchical fusion reconstruction network;

[0007] S2. Input the open set recognition task with N known classes into the trained hierarchical fusion reconstruction network to obtain the prediction results of the known classes and the latent features of hierarchical reconstruction. Then, input the prediction results of the known classes and the latent features of hierarchical reconstruction into the Openmax classifier to obtain the prediction results of the unknown classes of the open set recognition task with N known classes.

[0008] The beneficial effects of this invention are as follows: A SAR open-set recognition method based on hierarchical reconstruction latent representation learning is proposed to achieve known category recognition and unknown category detection of SAR images. First, latent reconstruction representations from each intermediate layer are learned and fused using HFRNet. Then, Openmax is used to correct the recognition score and calculate the probability of unknown scores, achieving efficient open-set recognition of SAR images. Finally, experimental results based on a real-world dataset demonstrate that this method has superior open-set recognition performance. Attached Figure Description

[0009] Figure 1 The algorithm flowchart of the method of this invention.

[0010] Figure 2 A schematic diagram comparing the input and output of the hierarchical fusion reconstruction network in a specific embodiment of the present invention;

[0011] (a) is the original SAR image, and (b) is an example of a reconstructed SAR image.

[0012] Figure 3 The specific embodiments of this invention employ receiver operating curves for unknown category detection.

[0013] Figure 4 The confusion matrix of the proposed open set identification method obtained through specific embodiments of the present invention.

[0014] Figure 5The specific embodiments of the present invention provide scores for different methods with varying degrees of openness. Detailed Implementation

[0015] This invention studies and designs a SAR open-set recognition method based on hierarchical reconstruction latent representation learning. The solution of this invention utilizes supervised learning prediction and a more generalized reconstructed representation, aided by hierarchical reconstruction latent representation learning, to achieve known class recognition and unknown class detection. For feature representation learning, a hierarchical fusion reconstruction network (HFRNet) is designed to learn fused latent reconstruction representations from intermediate layers and obtain preliminary closed-set recognition results. Then, Openmax is used to correct the closed-set recognition scores and calculate the probabilities of unknown scores, achieving effective open-set recognition of SAR images.

[0016] This invention is primarily verified using simulation experiments. All steps and conclusions were verified correctly on the Windows 10 operating system platform using PyCharm 2020.1.3. To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with accompanying drawings, further elaborates on the invention.

[0017] like Figure 1 As shown, the specific steps of this invention are as follows:

[0018] S1. Hierarchical Fusion Reconstruction Network. HFRNet utilizes a series of latent representations, namely z1, z2, z3, ..., z L , representing SAR images at different stages in the network, the information passed between reconstructed variables in the hierarchical structure can be represented as:

[0019]

[0020] Among them, f l h represents the convolution operation. l This represents a nonlinear dimensionality reduction process, where X is the input to the network. l G represents the input of the l-th layer. l This indicates an upsampling operation using a transposed convolutional layer. Indicates X l To restore the original dimension, with h l It is the opposite channel transformation operation. To fully utilize the latent representations at different stages, a fused latent representation is presented, expressed as:

[0021]

[0022] Where GAP represents global average pooling operation, and L is the number of network layers, which is set to 4 in this embodiment to achieve the best network performance. If the mapping function of HFRNet is expressed as F(·), then the final output of HFRNet can be formulated as:

[0023]

[0024] y c =[y c1 ,y c2 ,...,y cN ] T (4)

[0025] in, It is a reconstructed SAR image, y c Let y be the score vector for identifying the closed set, where y c1 This represents the probability of being identified as belonging to the first class. The open-set identification result is given by the open-set classifier. X is the original SAR image, representing the given potential fusion characterization. Figure 2 This is an example comparing the original image and the reconstructed image. Therefore, the total loss function of HFRNet can be expressed as:

[0026]

[0027] Where CE is cross-entropy, MSE is mean squared error, and y represents the true score vector. Using closed-set recognition task data of known classes provided by the MSTAR program, the network is trained using the above loss function, and iteration stops after reaching the required number of training iterations. In this embodiment, the number of iterations is 500.

[0028] S2 Open Set Classifier. For an open set identification task with N known classes, an Openmax classifier is used to obtain open set predictions based on the prediction results of the known classes and hierarchical reconstruction of latent features. For the m-th SAR image in the i-th class, the distance from the activation vector to the class center during open set testing can be expressed as:

[0029]

[0030] Wherein, y obtained in the previous step c , As input to the Openmax classifier μ represents the activation vector of the m-th SAR image in the i-th class. i Let X be the class center of class i, and let X be the average activation vector of the training data for class i. Then, the SAR image X is evaluated using the Weibull distribution. i The probability of belonging to the i-th class, and the corresponding weights for correcting the i-th closed set recognition score, can be expressed as:

[0031]

[0032] Where, ρ i =(m i ,η i R represents the Weibull distribution parameter of the i-th class, which is derived from the training data; α (i) = max(0, (α - rank(i)) / α) is used to apply a larger confidence discount to the categories with higher confidence, where rank(i) is the activation vector index sorted in descending order. Therefore, the output of the Openmax classifier, which is the open set recognition score y, is... o =[y o1 ,y o2 ,…,y oN ,y o(N+1) ] T Represented as:

[0033]

[0034] Among them, y ci ,y cn ∈y c =[y c1 ,y c2 ,…,y cN ] T , is the closed set recognition score vector from the previous step.

[0035] The experimental SAR images used to evaluate the proposed method were collected from the Moving and Stationary Target Acquisition and Identification (MSTAR) program. In the experiments, six classes were known (rocket launchers, 2S1; armored personnel carriers BMP2, BRDM2, BTR70, BTR60; bulldozers, D7) and four unknown classes (tanks, T62 and T72; trucks, ZIL131; air defense units, ZSU23 / 4).

[0036] To verify the effectiveness of the method proposed in this invention, the following was used: Figure 3 The average area under the receiver operating curve (ROC) is calculated as shown in Table 2, which is further visualized. Figure 4 , Figure 5 Table 2 shows the overall recognition performance of different methods under different open set complexities. The open set recognition performance results when there are two unknown classes in the test are shown in Table 2. Figure 3 The horizontal axis represents the false positive rate, the vertical axis represents the true positive rate, Ours represents the method of this invention, Openmax represents the existing Openmax classifier, and Softmax represents the existing Softmax classifier. Figure 4The horizontal axis represents the predicted label, the vertical axis represents the true label, and the unknown category represents the unknown class. Figure 5 The horizontal axis Openness represents the open set complexity, and the vertical axis Macro F1 represents the recognition performance.

[0037] Table 1 shows the open set recognition performance when there are two unknown classes in the test obtained by the specific embodiment of the present invention; in terms of open set recognition capability, when two unknown classes are placed in the test set, Table 2 shows the detection performance of the unknown classes of different methods of AUROC obtained by the specific embodiment of the present invention, and records four related indicators, including precision (PR), recall (RC), accuracy (ACC) and macro F1 score (MF1).

[0038] Table 1 shows the open set recognition performance when there are two unknown classes in the test.

[0039] Softmax 0.742 0.834 0.832 0.781 Openmax 0.859 0.836 0.838 0.844 This invention 0.893 0.857 0.859 0.869

[0040] Table 2. Detection performance of unknown classes using AUROC methods.

[0041] Softmax 0.751 Openmax 0.792 This invention 0.831

[0042] Regarding the distinction between known and unknown classes, Table 1 shows that the method of this invention achieves the best open-set recognition performance under all standards, with an accuracy of 85.9%. Table 2 also presents the average area under the receiver operating curve (AUROC), which represents the average detection performance of the unknown class. Among the three methods, the method of this invention achieves the highest AUROC value, demonstrating that it has the best ability to distinguish unknown classes. Figure 3 In the graph, the horizontal and vertical axes represent the true positive rate and the false positive rate, respectively. The true positive rate represents the proportion of successfully detected unknown class samples out of all unknown class samples, while the false positive rate represents the proportion of known class samples misclassified as unknown class samples. Clearly, the method of this invention achieves the best true positive rate compared to other methods.

[0043] In terms of open set recognition, Figure 4 The image shows the confusion matrix of the method of this invention, demonstrating its excellent open-set recognition performance. From... Figure 5 It can be inferred that as the openness increases, the recognition performance will naturally decrease because more unknown classes are added to the test set. Compared with the Openmax classifier, the method of this invention significantly improves the open set recognition performance by an average of 3%, proving the effectiveness of hierarchical reconstruction of latent representation learning.

[0044] Therefore, by using the hierarchical reconstruction of the latent representation provided by HFRNet, the information lost in supervised prediction can be supplemented, thereby further improving the SAR open set recognition performance of the Openmax-based method.

[0045] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A radar target open-set recognition method based on hierarchical reconstruction latent representation learning, characterized in that, include: S1. Use SAR images from closed-set recognition tasks of known classes to train the hierarchical fusion reconstruction network; The hierarchical fusion reconstruction network utilizes a series of latent representations, namely , representing SAR images reconstructed from different layers in the hierarchical fusion reconstruction network, and the information passed between reconstruction variables in different layers of the hierarchical fusion reconstruction network is represented as: ； in, This represents the convolution operation. This represents a nonlinear dimensionality reduction process. The hierarchical fusion reconstruction network represents the first Layer input, The hierarchical fusion reconstruction network represents the first The output of the layer, The hierarchical fusion reconstruction network represents the first Layer input, The hierarchical fusion reconstruction network represents the first The output of the layer, This indicates an upsampling operation using a transposed convolutional layer. Indicates to To restore the original dimension, and It is the opposite channel transformation operation; The mapping function of the hierarchical fusion reconstruction network is expressed as: The final output of the hierarchical fusion reconstruction network can be formulated as follows: ； in, It is a reconstructed SAR image. Identify the score vector for the closed set. As a potential characteristic of fusion, This is the original SAR image; The expression is: ； in, This indicates a global average pooling operation. It involves layered fusion and reconstruction of the network layers; S2. Input the open set recognition task with N known classes into the trained hierarchical fusion reconstruction network to obtain the prediction results of the known classes and the latent features of hierarchical reconstruction. Then, input the prediction results of the known classes and the latent features of hierarchical reconstruction into the Openmax classifier to obtain the prediction results of the unknown classes of the open set recognition task with N known classes.

2. The radar target open set recognition method based on hierarchical reconstruction latent representation learning according to claim 1, characterized in that, The loss function used during the training of the hierarchical fusion reconstruction network is expressed as: ； in, It's cross-entropy. Mean square error, Represents the true fraction vector.

3. The radar target open set recognition method based on hierarchical reconstruction latent representation learning according to claim 2, characterized in that, It also includes correcting the weights of the closed-set recognition scores to calculate the probability of unknown scores, thus achieving effective open-set recognition of SAR images.

4. The radar target open set recognition method based on hierarchical reconstruction latent representation learning according to claim 3, characterized in that, For the i-th closed set recognition score, the corrected weights are expressed as: ； in, This represents the distance from the activation vector to the class center. Denotes the Weibull distribution parameters of the i-th class. ; Indicates confidence level discount. , These are activation vectors sorted in descending order. This represents the cumulative distribution function of the Weibull distribution.

5. The radar target open set recognition method based on hierarchical reconstruction latent representation learning according to claim 4, characterized in that, The final open set recognition score vector output by the Openmax classifier is represented as: The open set recognition score is represented as: ； in, , Identify the score vector for the closed set.

Images