An entropy-aware meta-learning approach for automatic target recognition in open-set radar images

Abstract

The present invention discloses an entropy-aware meta-learning method for automatic target recognition of open-set radar images, which is applied to the field of target recognition and aims to solve the problem that the existing SAR automatic target recognition cannot effectively distinguish open-set targets. The present invention constructs a dynamic feature space based on known classes by designing meta-learning tasks. In order to further enhance the distinction between unknown classes and known classes, an entropy-aware loss function is designed to force the model to enhance the feature space, and finally achieve accurate SAR target image known class recognition and unknown class rejection in an open-set environment.

Description

Technical Field

[0001] The invention belongs to the field of target recognition, and in particular relates to a radar image target recognition technology. Background Art

[0002] As an indispensable microwave remote sensing system, synthetic aperture radar (SAR) is widely used in civil and military fields. SAR automatic target recognition (ATR) is one of its important applications. With the continuous development of deep learning in recent years, SAR ATR has also increasingly used deep learning to solve bottleneck problems. However, most SAR ATR methods are set in a closed set, that is, all targets to be detected are known class targets that have been trained and learned. When there are some targets of unknown categories, the existing methods will still assign these targets to known categories, which is likely to cause some potential risks. For example, when some invisible danger levels are classified as known safety levels, battlefield reconnaissance may collapse. Therefore, distinguishing invisible target classes is an urgent problem to be solved in practical applications, namely the open set recognition (OSR) problem.

[0003] The goal of OSR is to build a model that can simultaneously recognize known classes and reject targets of unknown classes. For example, the paper "S. Dang, Z. Cao, Z. Cui, and Y. Pi, Open set sar target recognition using class boundary extracting, in 2019 6th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR). IEEE, 2019, pp. 1–4." uses class boundaries to train an open set recognition or outlier detection model, and the proposed O-SAR method performs well in accurately rejecting unknown targets. However, even though these existing methods have some understanding of OSR in SAR ATR, they are mainly based on hand-crafted features or prior knowledge, and the recognition performance also depends mainly on the quality of these prior information. In addition, most of them only construct a static feature space based on the entire closed dataset, and design hand-crafted features to reject limited open classes, which is not always effective for the actual OSR problem. Summary of the invention

[0004] To solve the above technical problems, the present invention proposes an entropy-aware meta-learning method for automatic target recognition in open-set radar images. By designing meta-learning tasks, a dynamic feature space is constructed based on known classes. In order to further enhance the distinction between unknown classes and known classes, an entropy-aware loss function is designed to force the model to enhance the feature space, ultimately achieving accurate known class recognition and unknown class rejection of SAR target images in an open-set environment.

[0005] The technical solution adopted by the present invention is: an entropy-aware meta-learning method for automatic target recognition of open-set radar images, comprising:

[0006] S1. Obtain a radar image dataset, and divide the radar image dataset into a meta-training set and a meta-test set;

[0007] S2, meta-training stage, by randomly selecting known classes and unknown classes, the training subset and test subset of the meta-learning task are obtained; the meta-learner is trained based on the training subset of each meta-learning task;

[0008] S3. Apply the trained meta-learner to the test subset of each meta-learning task for SAR target recognition.

[0009] Beneficial effects of the present invention: The method of the present invention first constructs a feature space of dynamically allocated known classes through meta-learning tasks, and forces the feature space to distinguish between unknown classes and known classes, thereby rejecting all other classes that do not belong to known classes under open set conditions. At the same time, the feature space of the model is enhanced by using the designed entropy-aware loss function, which can more effectively and robustly distinguish between known classes and unknown classes;

[0010] The present invention constructs a dynamic feature space, which can accurately reject unknown classes while dynamically identifying known classes. Experiments on the moving and stationary target acquisition and recognition (MSTAR) dataset show the excellent performance of the invention in open-set radar image automatic target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is the overall framework of the method of the present invention;

[0012] Figure 2 The specific structure of the method of the present invention and the details of the three parts of the entropy-aware loss function. DETAILED DESCRIPTION

[0013] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0014] Step 1: Divide the available radar image dataset into a meta-training set and a meta-test set. In the meta-training stage, the meta-training set is used for task-based meta-training. In meta-training, by randomly selecting known classes and unknown classes, the meta-learner faces many different tasks and is optimized based on the performance of these tasks, ultimately having the ability to identify known classes and reject unknown classes. In this embodiment, the available radar image dataset is a dataset that the application party already has or is open source in the application scenario.

[0015] Taking the meta-learning task as the basic unit of the training and testing phases, a meta-training dataset D is defined in the process mtr and a meta-test dataset D mte ,like Figure 1 As shown. Meta-training dataset D mtr In practical applications, it is regarded as a closed data set. mtr The model is trained on the open dataset, and it is expected that in the test phase it can recognize objects of known classes in the open dataset and reject objects of unknown classes outside the closed dataset. The open dataset refers to the test images, that is, the dataset containing images of unknown categories.

[0016] For all tasks sampled in the meta-training phase, the dataset can be expressed as Where C represents the set of available supervised image data classes (closed set), N tr is the number of all tasks in the meta-training phase, represents the training subset of the i-th task in the meta-training dataset, represents the test subset of the i-th task in the meta-training dataset. For each task in the meta-training phase, A part of the classes is randomly selected as the known class C closed , and set other classes to unknown classes U open Among them, it is known that class C closed and unknown class U open Two conditions need to be met: and In short, the training subset From the known class C closed Sampling, test subset T i tr All classes from the meta-training dataset Sampling obtained.

[0017] For the i-th task, the meta-training phase is divided into two steps. Given a meta-learner f θ (θ represents the parameters of the meta-learner) and a suitable loss L, the first step is to train the subset To optimize, the operation is as follows:

[0018]

[0019] in Represents the training subset S tr The corresponding category label (such as the category of SAR target vehicle images in MSTAR: infantry fighting vehicle (BMP2), armored transport vehicle (BTR70), tank (T72), etc., f′ θ Represents the entire training subset S tr The optimal meta-learner under .

[0020] The second step is to test the subset T tr Optimize, based on the f′ obtained from the above training θ Further update to get the best meta-learner for the i-th task

[0021]

[0022] in Represents the test subset T tr The corresponding category label.

[0023] Step 2: Optimize the model based on the designed entropy-aware loss function, including meta-cross entropy loss, entropy distance loss and open set loss. Figure 2 As shown, the specific network structure of the method of the present invention and the details of the three loss functions designed are shown. In short, the present invention first quotes the Swin Transformer network structure (derived from the document "Liu Z, LinY, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shiftedwindows, Proceedings of the IEEE / CVF international conference on computervision. 2021: 10012-10022."), and then inputs the original radar image data into the network to map the sample image to a higher-dimensional space, namely the feature space. The loss function is then used to reversely update the model, reduce the loss, and obtain a better feature expression to achieve subsequent accurate recognition.

[0024] For a given meta-training dataset As shown in formula (1), in S tr The meta-learner f′ trained on θ , where the entropy-aware loss function L is applied ea As the loss function L. ea The settings are as follows:

[0025] L ea =λ1Lmeta-ce +λ2L entropy +λ3L open (3)

[0026] Where L meta-ce is the meta-cross entropy loss, L entropy is the entropy distance loss, L open is an open set loss, λ1, λ2, and λ3 are weighted parameters, and the value range of λ1, λ2, and λ3 is [0,1].

[0027] Meta-cross entropy loss L meta-ce The design aims to maximize the feature distance entropy of known classes to help the model build the basic feature space, which provides the model with the basic ability to identify known classes. Based on the test subset T tr The meta-cross entropy loss L meta-ce The definition is as follows:

[0028]

[0029] Among them C closed represents the known class selected in this task, and y j ∈C closed In addition, in the calculation, if the input sample x i If (the i-th SAR image) belongs to class j, then yj = 1, otherwise yj = 0. j |x i )=sf(L2(-d(v(x i ),a j ))) represents the input x i The probability of the target image being identified as class j by the model, where sf(·) represents the SoftMax function, L2(·) represents the L2 normalization process, and d(v(x i ),a j ) represents v(x i ) and a j The distance between them. v(x i )=GAP(net(x i )) refers to the input image sample x i The final feature map is extracted, where GAP(·) refers to the global average pooling operation and net(·) refers to Figure 2The operations of each layer of the network structure shown (for specific operations, please refer to the document "Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows, Proceedings of the IEEE / CVF international conference on computer vision. 2021: 10012-10022."). a j Refers to the prototype of the j-type target image (the calculation refers to the prototype definition in the literature "J.Snell, K.Swersky, and R.Zemel, Prototypical networks for few-shot learning, Advances in neural information processing systems, vol.30, 2017.").

[0030] Entropy distance loss L entropy The design of L is to minimize the feature distance entropy of the unknown class relative to the known class, which provides the model with the basic ability to reject unknown classes. In short, L entropy By calculating and reducing the entropy of unknown classes assigned to known classes, the feature distance between known and unknown classes is enlarged, such as Figure 2 As shown. Based on the open set U open Sample x i Entropy distance loss L entropy The definition is as follows:

[0031]

[0032] Open set loss L open The design of forces the constructed feature space to distinguish between known classes and unknown classes. Figure 2 As shown in the figure, after the feature map extracted by the network passes through the global average pooling layer (GAP), the feature passes through the dense layer recognizer of SoftMax, and the open set loss is calculated as follows:

[0033]

[0034] In summary, the meta-learner can make the feature distance entropy between known classes generally larger than the feature distance entropy of unknown classes relative to known classes through the three components of the entropy-aware loss function, and realize the model's distinction between unknown classes by judging the size of the feature distance entropy. Finally, it promotes the model to achieve effective and robust distinction between known and unknown classes, while accurately identifying known classes.

[0035] Through the above loss and optimization process, the best meta-learner f for the i-th task can be optimized θ * , and use it as the initial meta-learner for the (i+1)th task to iteratively complete the meta-training phase.

[0036] Step 3: In the meta-test phase, accurate SAR target recognition under open set conditions is achieved, and the performance of the method of the present invention for the OSR problem is evaluated.

[0037] After the meta-learner is continuously optimized under N meta-training tasks, it will be evaluated in the meta-testing phase. The meta-testing dataset is represented as Its training subset S te Randomly sample from the closed set C, test subset T te Sampled from all classes (C∪U) containing both known and unknown labels. Meta-learner By training the subset S te Generate the final optimal model f″ θ :

[0038]

[0039] in is the training subset S te The corresponding label.

[0040] So far, we have obtained the final meta-learner f″ θ , which will be applied to the test subset T te To complete the final SAR target recognition. Set the three categories of BMP2, BTR70 and T72 in the MSTAR dataset as known classes, and the remaining seven classes as unknown classes. Table 1 shows the open set recognition performance comparison between the method of the present invention and other open set recognition methods under different recognition indicators. Among them, the true positive rate is the proportion of all true known class samples currently predicted as true known class samples in the known class; the negative positive rate is the proportion of all true unknown class samples currently wrongly predicted as true unknown class samples in the known class; the recall rate indicates the probability that all known class samples can be correctly identified; the precision rate indicates the probability of correct identification in the predicted known class samples; the accuracy rate gives the average probability of correctly identifying each known class and accurately rejecting the unknown class.

[0041] Table 1 Comparison results between the proposed method and other open set SAR target recognition methods

[0042]

[0043] In this embodiment, the present invention dynamically constructs a feature space for a closed set through a meta-learning task, and at the same time assists in designing an entropy-aware loss function to enhance the ability to identify known classes and reject unknown classes, so that the model can accurately predict the probability distribution of known classes while also giving a value to predict whether the test image belongs to any known class, and finally realize open set SAR target recognition. Experiments have shown that the present invention exhibits excellent performance in the MSTAR dataset.

[0044] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An entropy-aware meta-learning method for automatic target recognition in open-set radar images, characterized in that: include: S1. Obtain a radar image dataset, and obtain a meta-training dataset and a meta-testing dataset according to the radar image dataset; The meta-training data set and the meta-testing data set each include a corresponding training subset and a test subset; S2, meta-training stage, for each task in the meta-training stage, a part of the known classes in the radar image data is randomly selected as the known class C closed , and set the remaining classes in the known class set of radar image data to unknown classes U open ; Among them, it is known that class C closed and unknown class U open satisfy: and D mtr Represents the meta-training dataset, C represents the set of available supervised image data classes, and the meta-learner is trained based on the training subset and test subset of the meta-training dataset of each meta-learning task; in step S2, the loss function used in the meta-training stage is: L ea =λ1L meta-ce +λ2L entropy +λ3L open Among them, L meta-ce is the meta-cross entropy loss, L entropy is the entropy distance loss, L open is the open set loss, λ1, λ2 and λ3 are weighting parameters; Meta-cross entropy loss L meta-ce The definition is as follows: Among them C closed represents the known class selected in this task, and y j ∈C closed , x i Represents an open set U open Samples of Entropy distance loss L entropy The definition is as follows: The open set loss is as follows: S3, meta-testing stage, the meta-learner in step S2 generates the final optimal model based on the training subset in the meta-testing dataset; S4. Apply the final optimal model obtained in step S3 to the test subset in the meta-test data set for SAR target recognition.

2. The entropy-aware meta-learning method for automatic target recognition in open-set radar images according to claim 1, characterized in that: The training subset in the meta-training dataset is from the known class C closed The test subset in the meta-training dataset is sampled from the known class set of radar image data.

3. The entropy-aware meta-learning method for automatic target recognition in open-set radar images according to claim 2, characterized in that: The training subset of the meta-test dataset is sampled from the known class set of radar image data, and the test subset T te The samples are obtained from the total set of all classes including the known class set and the unknown class set of radar image data.

Images