SAR Image Open-Set Target Recognition Method Combining Hard Negative Generation and Learning

Abstract

The present invention discloses a method for open-set recognition of SAR image targets combining the generation and learning of hard-to-separate samples, and its implementation steps are as follows: generating a training set and a test set; constructing a convolutional neural network and training it in combination with hard-to-separate samples; setting a rejection threshold for the open-set recognition convolutional neural network according to the image categories in the training set; obtaining the predicted probability distribution of the SAR image target to be recognized; identifying the samples with the maximum value greater than the rejection threshold in the predicted probability distribution as the category corresponding to the maximum value, and rejecting the samples with the maximum value less than the rejection threshold as unknown target categories. The present invention solves the problems of poor data adaptability and sensitive setting of rejection thresholds in the prior art, and has the advantages of high-precision recognition of known target categories in the library and rapid and accurate discovery of unknown target categories outside the library.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar communication, and further relates to a method for open-set recognition of synthetic aperture radar (SAR) targets combining the generation and learning of hard-to-separate samples in the technical field of radar automatic target recognition. The present invention can be used to identify the types of SAR image targets in an open scenario of a radar jamming environment. Background Art

[0002] Radar Automatic Target Recognition (RATR) is a typical open-set recognition task, which faces cooperative and non-cooperative targets with a large number of categories and complex models. It is very difficult to establish a complete target recognition library during the training stage, that is, there are unknown-class targets in the test stage. Traditional pattern recognition techniques are designed under the closed-set assumption, that is, it is assumed that the set of class labels that appear in the training samples contains all the class labels of the samples to be recognized, and there are no unknown-class targets. In a real open environment, when an out-of-library unknown-class target enters the model, the closed-set recognition model will forcefully misclassify the unknown-class target as a certain in-library category, which greatly limits the application of the model in the open environment. Therefore, a classifier that can recognize / reject unknown-class targets while maintaining the recognition performance for known-class targets is expected. In radar automatic target recognition, for the input known-class samples, the output is a specific category, and for the input unknown-class samples, it is rejected as an unknown-class category. Perceiving unknown-class targets in an open environment and correctly and adaptively detecting the existence of unknown-class targets and recognizing known-class targets are the premise and foundation for radar open-set recognition.

[0003] Zeng Zhiqiang et al. proposed an open-set recognition method for SAR image targets that combines relative entropy and relative position angle measurement in their published paper "Unknown SAR Target Identification Method Based on Feature Extraction Network and KLD–RPA Joint Discrimination" (Remote Sensing, 2021, 7). This method consists of a feature extractor and a classifier. The feature extractor inputs the images of the training set and outputs 128-dimensional high-dimensional features. The classifier inputs the 128-dimensional high-dimensional features and outputs the predicted probability distribution corresponding to the features. The implementation steps of this method are as follows: First, all the images of the training set are input into the feature extractor to obtain high-dimensional features, and the high-dimensional features are input into the classifier to obtain the predicted probability distribution, and the cross-entropy loss between the predicted probability distribution and the true label is minimized; Second, calculate the mean of the 128-dimensional features of the images of different classes in the training set and the relative angle range of the corresponding features in the test set compared with the training set features. In the test stage, the image target to be recognized is input into the feature extractor, and 128-dimensional high-dimensional features are output. When the relative entropy of the 128-dimensional high-dimensional features and the mean of the features of each class in the training set is greater than the preset relative entropy threshold, or the relative angle range of the features compared with the training set features is greater than the preset relative angle threshold, the model will reject the image target to be recognized as an unknown target class, otherwise it will be sent to the classifier to obtain the predicted probability distribution and be recognized as the class corresponding to the maximum probability prediction value. The disadvantage of this method is that two appropriate thresholds need to be preset with the help of the test set image information in the training stage. Since the test set image categories cannot be obtained in advance in engineering practice, the rejection threshold setting strategy of this method is highly sensitive and has poor data adaptability, far from meeting the requirements of engineering practice in real open scenarios.

[0004] Xidian University disclosed a radar target open set recognition method in its patent document "A Radar High-Resolution Range Profile Open Set Recognition Method and Device" (Patent Application No.: CN202111199838.4, Publication No.: CN 114137518 A). This method uses convolutional neural network technology to construct prototype vectors for each category to enhance intra-class aggregation, and at the same time combines the primary features of each layer to obtain higher-level features for target recognition. Therefore, the recognition rate of known target classes in the invention has been significantly improved. In the test stage, when the maximum predicted probability value of the target to be recognized is less than the artificially set threshold, the model rejects the target to be recognized as an unknown class target; otherwise, it is recognized as the class corresponding to the maximum predicted probability value. The disadvantage of this method is that only the recognition performance of known target classes is concerned in the training stage, the open space risk of the model cannot be effectively reduced, it is difficult to establish a good rejection interface, and serious confusion will occur when directly applied to real open scenarios, far from meeting the requirements of recognition performance in the engineering practice process. Summary of the Invention

[0005] The object of the present invention is to propose a SAR image target open set recognition method combining the generation and learning of difficult-to-separate samples in view of the above deficiencies of the existing technology, aiming to solve the problems of poor data adaptability and sensitive rejection threshold setting in the existing technology.

[0006] The idea to achieve the object of the present invention is that the present invention uses the training set images and noise sequences to calculate the features of difficult-to-separate samples that are easy to be confused in the feature space, generates high-quality difficult-to-separate images in the data space, and forms difficult-to-separate samples that can represent the statistical characteristics of unknown target classes in the actual open space. The learning of difficult-to-separate samples by the network enhances the perception ability of unknown target classes and alleviates the problem of poor data adaptability caused by the inability to reduce the open space risk in the existing technology. The present invention automatically determines the rejection threshold according to the statistical characteristics of the training set images, alleviating the problem of high sensitivity caused by artificially setting the rejection threshold in the existing technology. The designed difficult-to-separate sample generation and learning method of the present invention overcomes the problems of poor data adaptability and sensitive rejection threshold setting in the existing technology, and realizes the accurate recognition of known target classes in the library and the accurate rejection of unknown target classes outside the library.

[0007] The technical solution adopted by the present invention includes the following steps:

[0008] Step 1, generate a training set:

[0009] Each of the 2746 images with a radar working pitch angle of 17° in the sample set is cropped into an image of 64×64 pixels. All the selected images contain 10 categories. Within the range of class labels [1, 10], the target category of each image is labeled, and then all the cropped images and the corresponding class labels are combined to form a training set;

[0010] Step 2, construct an open-set recognition convolutional neural network:

[0011] Step 2.1, construct a generator sub-network:

[0012] Build a generator sub-network composed of five transposed convolutional layers in series. Set the number of convolutional kernels of the first to fourth transposed convolutional layers to 1024, 512, 256, 128, and 3 in sequence, set the size of the convolutional kernels to 4×4, set the stride of the convolutional kernels to 1, 2, 2, 2, and 2 in sequence, set the padding method to equal-size padding, and set the bias to 0;

[0013] Step 2.2, construct a discriminator sub-network:

[0014] Build a discriminator sub-network composed of four convolutional layers in series. Set the number of convolutional kernels of the first to fourth convolutional layers to 128, 256, 512, and 1024 in sequence, set the size of the convolutional kernels to 4×4, set the stride of the convolutional kernels to 1, 1, 1, and 0 in sequence, set the padding method to equal-size padding, and set the bias to 0;

[0015] Step 2.3, construct a classifier sub-network:

[0016] Construct a classifier sub-network composed of a group of convolutional layers and a fully connected layer in series, where a group of convolutional layers is composed of nine convolutional layers in series; set the number of convolutional kernels of the first to ninth convolutional layers to 64, 64, 128, 128, 128, 128, 128, 128, and 128 in sequence, set the size of the convolutional kernels to 3×3, set the stride of the convolutional kernels to 1, 1, 2, 1, 1, 2, 1, 1, and 2 in sequence, set the padding method to equal-size padding, and set the bias to 0; set the parameters of the fully connected layer to 128×K, where K represents the total number of categories of training samples, and set the bias to 0;

[0017] Step 2.4, connect the generator sub-network and the discriminator sub-network in parallel and then connect them in series with the classifier sub-network to form an open-set recognition convolutional neural network;

[0018] Step 3, train the open-set recognition convolutional neural network by combining hard-to-separate samples:

[0019] Step 3.1, Input the noise sequence randomly sampled from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated images output after passing through the generator sub-network are input into the discriminator sub-network, and the discriminative confidence scores of each noise-generated image are output. Each image in the training set is input into the open-set recognition convolutional neural network, and the discriminative confidence scores of each image are output after passing through the discriminator sub-network. Using the cross-entropy loss function, calculate the loss value between the discriminative confidence score of each image and the domain label corresponding to this image to obtain the total loss function of the discriminator sub-network. Use the Adam optimization algorithm to update the current parameters of the discriminator sub-network;

[0020] Step 3.2, Input the noise sequence randomly sampled again from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated images output after passing through the generator sub-network are input into the discriminator sub-network, and the discriminative confidence scores of each noise-generated image are output. After passing through the classifier sub-network, the predicted probability distribution of each noise-generated image is output. Using the cross-entropy loss function, calculate the loss value between the predicted probability distribution of each noise-generated image and the class label corresponding to this image, and calculate the loss value between the discriminative confidence score of each noise-generated image and the domain label to obtain the total loss function of the generator sub-network. Use the Adam optimization algorithm to update the current parameters of the generator sub-network;

[0021] Step 3.3, Input each image in the training set into the open-set recognition convolutional neural network. After passing through the classifier sub-network, the known target class features of each image are output, and the hard-to-separate sample features corresponding to each image are calculated by the hard-to-separate sample feature generation function using gradient descent;

[0022] Step 3.4, Input the noise sequence randomly sampled again from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated images are output after passing through the generator sub-network. After the noise-generated images pass through the discriminator sub-network, hard-to-separate images with higher discriminative confidence scores are generated according to the discriminative confidence scores. After passing through the classification sub-network, the hard-to-separate image features of each hard-to-separate image are output. The hard-to-separate image features and the hard-to-separate sample features together form the hard-to-separate feature set, and the known target class features form the training feature set;

[0023] Step 3.5, Input the training feature set and the hard-to-separate feature set into the open-set recognition convolutional neural network. After passing through the classifier sub-network, the predicted probability distribution corresponding to each feature is output. Using the cross-entropy loss function, calculate three loss functions respectively. Calculate the loss value between each feature in the training feature set and the class label corresponding to this feature, calculate the loss value between each feature in the training feature set and the smoothed label corresponding to this feature, and calculate the loss value between each feature in the hard-to-separate feature set and the uniform distribution to obtain the total loss function of the classifier sub-network. Use the SGDM optimization algorithm to update the current parameters of the classifier sub-network;

[0024] Step 3.6, repeat Steps 3.1 to 3.5 to iteratively update the parameters of the generator subnet, discriminator subnet, and classifier subnet in the open-set recognition convolutional neural network for the number of training times, and obtain the trained open-set recognition convolutional neural network;

[0025] Step 4, set the rejection threshold of the open-set recognition convolutional neural network;

[0026] Step 5, obtain the predicted probability distribution of the SAR image target to be recognized;

[0027] In the same way as in Step 1, crop the SAR image to be recognized to obtain the cropped SAR image target, input the cropped SAR image target into the open-set recognition convolutional neural network, and output the corresponding predicted probability distribution after passing through the classifier subnet;

[0028] Step 6, determine whether the maximum value in the predicted probability distribution of the SAR image target to be recognized is greater than the rejection threshold. If so, execute Step 7; otherwise, execute Step 8;

[0029] Step 7, select the class corresponding to the maximum score in the predicted probability distribution of the SAR image target as the recognition result for output;

[0030] Step 8, determine the SAR image target as an unknown target class for output.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] First, the present invention calculates and generates difficult-to-separate samples that characterize the statistical characteristics of difficult-to-separate unknown target classes in the actual open space. Through the learning of the network, the perception ability of unknown target classes is improved, overcoming the deficiency of poor data adaptability in the prior art, and enabling the present invention to have good generalization ability.

[0033] Second, the present invention automatically determines the rejection threshold according to the statistical characteristics of the training set images, overcoming the deficiency of high sensitivity caused by artificially presetting the threshold in the prior art, enabling the present invention to accurately identify known target classes in the library while accurately rejecting unknown target classes outside the library. Brief Description of the Drawings

[0034] Figure 1 is the implementation flowchart of the present invention;

[0035] Figure 2 is the overall model schematic diagram of the present invention;

[0036] Figure 3 Comparison diagram of the simulation experiment results of the present invention. Detailed Embodiments

[0037] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0038] Reference Figure 1 , the implementation steps of specific embodiments of the present invention will be further described in detail.

[0039] Step 1, generate a training set and a test set.

[0040] Step 1.1, in the embodiment of the present invention, all 5172 images are selected from the SAR vehicle target dataset MSTAR (moving and stationary target acquisition) to form a sample set;

[0041] Step 1.2, each of the 2746 images with a radar working pitch angle of 17° in the sample set is cropped into an image of 64×64 pixels. All the selected images contain 10 categories. Within the range of category labels [1, 10], the target category of each image is labeled, and then all the cropped images and the corresponding category labels are combined to form a training set;

[0042] Step 1.3, each of the 2426 images with a radar working pitch angle of 15° in the sample set is cropped into an image of 64×64 pixels. All the selected images contain 10 categories. Within the range of category labels [1, 10], the target category of each image is labeled, and then all the cropped images and the corresponding category labels are combined to form a test set;

[0043] Step 2, construct an open-set recognition convolutional neural network:

[0044] Combined with Figure 1 , the generator sub-network, discriminator sub-network, and classifier sub-network of the constructed open-set recognition convolutional neural network will be described in detail.

[0045] Step 2.1, construct a generator sub-network:

[0046] Build a generator sub-network composed of five deconvolution layers in series. The number of convolutional kernels of the first to fourth deconvolution layers is set to 1024, 512, 256, 128, and 3 in sequence, the convolutional kernel size is set to 4×4, the convolutional kernel stride is set to 1, 2, 2, 2, and 2 in sequence, the padding method is set to equal-size padding, and the bias is set to 0. The input dimension of the generator sub-network is 100-dimensional random noise, and the output is the noise-generated image generated by the generator sub-network;

[0047] Step 2.2, construct a discriminator sub-network:

[0048] Build a generator sub-network composed of four convolutional layers in series. Set the number of convolutional kernels of the first to fourth convolutional layers to 128, 256, 512, and 1024 in sequence, set the size of the convolutional kernels to 4×4, set the convolutional kernel strides to 1, 1, 1, and 0 in sequence, set the padding method to equal-size padding, and set the bias values to 0. The discriminator sub-network inputs noise to generate images or images from the training set, and outputs the discriminant confidence scores calculated by the discriminator sub-network;

[0049] Step 2.3, construct a classifier sub-network:

[0050] Construct a classifier sub-network composed of a group of convolutional layers and a fully connected layer in series. Among them, a group of convolutional layers is composed of nine convolutional layers in series; set the number of convolutional kernels of the first to ninth convolutional layers to 64, 64, 128, 128, 128, 128, 128, 128, and 128 in sequence, set the size of the convolutional kernels to 3×3, set the convolutional kernel strides to 1, 1, 2, 1, 1, 2, 1, 1, and 2 in sequence, set the padding method to equal-size padding, and set the bias values to 0; set the parameters of the fully connected layer to 128×K, where K represents the total number of categories of training samples, and set the bias values to 0. The classifier sub-network inputs noise to generate images or images from the training set, and outputs the predicted probability distribution calculated by the classifier sub-network;

[0051] Step 2.4, connect the generator sub-network and the discriminator sub-network in parallel and then connect them in series with the classifier sub-network to form an open-set recognition convolutional neural network;

[0052] Step 3, train the open-set recognition convolutional neural network in combination with difficult-to-classify samples:

[0053] Step 3.1, input the noise sequence randomly sampled from the standard normal distribution N(0,1) into the open-set recognition convolutional neural network. The noise-generated images output after passing through the generator sub-network, and the discriminant confidence scores of each noise-generated image are output after passing through the discriminator sub-network. Among them, the total number of noise sequences is equal to the total number of images in the training set; input each image in the training set into the open-set recognition convolutional neural network, and the discriminant confidence scores of each image are output after passing through the discriminator sub-network. Use the cross-entropy loss function to calculate the loss value between the discriminant confidence score of each image and the domain label corresponding to the image to obtain the total loss function of the discriminator sub-network; use the Adam optimization algorithm to update the current parameters of the discriminator sub-network.

[0054] The total loss function of the discriminator sub-network:

[0055]

[0056] Among them, L Ddenotes the total loss function of the discriminator sub-network, \(N_0\) denotes the total number of training set images and the total number of noise sequences, \(\Sigma\) denotes the summation operation, \(i\) denotes the image serial number of the training set images, \(j\) denotes the noise serial number of the noise sequences, \(\log\) denotes the logarithm operation with base 2, \(D(x (i) )\) denotes the discriminative confidence score output after inputting the \(i\)-th image in the training set images into the open-set recognition convolutional neural network and passing through the discriminator sub-network, \(D(G(z (j) ))\) denotes the discriminative confidence score output after inputting the \(j\)-th noise in the noise sequences into the open-set recognition convolutional neural network, generating a noise-generated image through the generator sub-network, and passing through the discriminator sub-network;

[0057] Step 3.2, input the noise sequence randomly sampled again from the standard normal distribution \(N(0, 1)\) into the open-set recognition convolutional neural network, the noise-generated image output after passing through the generator sub-network, the discriminative confidence score of each noise-generated image output after passing through the discriminator sub-network, and the predicted probability distribution of each noise-generated image output after passing through the classifier sub-network. Among them, the total number of noise sequences is equal to the total number of training set images; use the cross-entropy loss function to calculate the loss value between the predicted probability distribution of each noise-generated image and the class label corresponding to this image, and calculate the loss value between the discriminative confidence score of each noise-generated image and the domain label, to obtain the total loss function of the generator sub-network; use the Adam optimization algorithm to update the current parameters of the generator sub-network.

[0058] The total loss function of the generator sub-network:

[0059]

[0060] where \(L G \) denotes the total loss function of the generator sub-network, \(\lambda\) denotes the training hyperparameter set to 0.1, \(K\) denotes the total number of categories of the training set images, \(C(G(z (j) ))\) denotes the predicted probability distribution output after inputting the \(j\)-th noise in the noise sequences into the open-set recognition convolutional neural network, generating a noise-generated image through the generator sub-network, and passing through the generator sub-network;

[0061] Step 3.3, input each image of the training set into the open-set recognition convolutional neural network, the known target class features of each image are output after passing through the classifier sub-network, and the hard-to-separate sample features corresponding to each image are calculated by the hard-to-separate sample feature generation algorithm of gradient descent.

[0062] The steps of the hard-to-separate sample feature generation algorithm of gradient descent are as follows:

[0063] The first step, according to the following formula, calculate the loss function of all hard-to-separate sample feature generations:

[0064]

[0065] Among them, L U represents the loss function for generating hard-to-separate sample features, represents the class smoothed label corresponding to the i-th image in the training set images. In the embodiments of the present invention, α is set as a parameter of 0.5, represents a uniform distribution vector, f (i) represents inputting the i-th image in the training set images into the open-set recognition convolutional neural network, and outputting the corresponding known target class features after passing through the classifier sub-network, represents obtaining the corresponding hard-to-separate sample features after passing the i-th image in the training set images through the hard-to-separate sample feature generation function of gradient descent.

[0066] Step 2: Use the gradient descent method to minimize the loss function for generating hard-to-separate sample features, calculate the result of gradient descent; update the hard-to-separate sample feature parameters until the loss function for generating hard-to-separate sample features converges, and obtain the corresponding hard-to-separate sample features.

[0067] The gradient descent method minimizes the loss function for generating hard-to-separate sample features as follows:

[0068]

[0069] Among them, the hard-to-separate sample features are initialized as the corresponding known target class features; represents the update value of the i-th hard-to-separate sample feature; η represents the learning rate of the gradient descent method. In the embodiments of the present invention, 0.01 is set;

[0070] Step 3.4: Input the noise sequence randomly sampled again from the standard normal distribution N(0, 1) into the open-set recognition convolutional neural network, and output the noise-generated images after passing through the generator sub-network. Among them, the total number of noise sequences is equal to three times the total number of training set images; after the noise-generated images pass through the discriminator sub-network, hard-to-separate images with higher discriminant confidence scores are generated according to the discriminant confidence scores. Among them, the total number of hard-to-separate images is equal to the total number of training set images; after the hard-to-separate images pass through the classification sub-network, the hard-to-separate image features of each hard-to-separate image are output. The hard-to-separate image features and the hard-to-separate sample features jointly form the hard-to-separate feature set, and the known target class features form the training feature set;

[0071] Step 3.5: Input the training feature set and the hard-to-separate feature set into the open-set recognition convolutional neural network. After passing through the classifier sub-network, the predicted probability distribution corresponding to each feature is output. Using the cross-entropy loss function, three loss functions are calculated respectively. Calculate the loss value between each feature in the training feature set and the class label corresponding to this feature, calculate the loss value between each feature in the training feature set and the smoothed label corresponding to this feature, and calculate the loss value between each feature in the hard-to-separate feature set and the uniform distribution, to obtain the total loss function of the classifier sub-network; use the SGDM optimization algorithm to update the current parameters of the classifier sub-network.

[0072] The total loss function of the classifier sub-network is as follows:

[0073]

[0074] where, L C represents the total loss function of the classifier sub-network, k represents the serial number of the feature in the training feature set, N1 represents the total number of the hard-to-separate feature set, and l represents the serial number of the feature in the hard-to-separate feature set.

[0075] The hard-to-separate sample generation and learning method adopted by the present invention enables the network to rely on the training set images and noise sequences to generate high-quality hard-to-separate images in the data space, and calculate the hard-to-separate sample features that are easily confused in the feature space; through the learning of the hard-to-separate feature set, the network's perception ability for unknown target classes is enhanced, and the open space risk is reduced;

[0076] Step 3.6: Repeat Steps 3.1 to 3.5, and iteratively update the parameters of the generator sub-network, discriminator sub-network, and classifier sub-network in the open-set recognition convolutional neural network 200 times to obtain the trained open-set recognition convolutional neural network;

[0077] In the embodiment of the present invention, the number of model training times is set to 200 times, and the Adam optimizer and SGMD optimizer are used during the training process. Among them, the Adam first-order exponential decay factor is 0.9, the second-order exponential decay factor is 0.999, and the learning rate is set to 0.0001; the SGDM momentum is set to 0.9, the decay rate is set to 0.0005, the initial value of the learning rate is 0.1, and when the training times reach 60, 120, and 180, the learning rate is multiplied by 0.1.

[0078] Step 4: Set the rejection threshold τ of the open-set recognition convolutional neural network as τ=(1 - α)+α / K; where, τ represents the rejection threshold.

[0079] The present invention uses the maximum value corresponding to the smoothed label in the training stage as the rejection threshold of the network, which alleviates the problem of high sensitivity in artificially setting the rejection threshold;

[0080] Step 5: Test the open-set recognition convolutional neural network:

[0081] Step 5.1: Input each test image in the test set into the classifier sub-network of the trained open-set recognition convolutional neural network to output the predicted probability distribution of each test image.

[0082] Step 5.2: Compare the relative magnitudes between the maximum value of the predicted probability distribution of the test image and the rejection threshold. If the maximum predicted probability value is greater than the rejection threshold, the test image to be recognized is identified as the category corresponding to the maximum predicted probability value; otherwise, it is rejected as an unknown target category.

[0083] The effects of the present invention will be further described below in conjunction with simulation experiments.

[0084] 1. Simulation conditions:

[0085] The hardware platform for the simulation experiment of the present invention is as follows: The processor is an Intel i9 10900CPU with a main frequency of 2.80GHz and a memory of 16GB.

[0086] The software platform for the simulation experiment of the present invention is as follows: Windows 10 operating system, MATLAB R2021a, and python3.7.

[0087] The dataset used in the simulation experiment of the present invention is the SAR vehicle target dataset MSTAR (moving and stationary target acquisition). The data in this dataset are SAR images of various target vehicles collected by a high-resolution spotlight synthetic aperture radar. The imaging time is in the mid-1990s. The image size is 128×128 pixels. The image contains SAR image target data of seven types of targets, such as BTR60, 2S1, and BRDM2, with 9 different pitch angles. The image format is mat. In the simulation experiment of the present invention, all images with a pitch angle of 17° in the dataset are used to form a training sample set, with a total of 2746 images. All images with a pitch angle of 15° in the dataset are used to form a test sample set, with a total of 2426 images.

[0088] 2. Simulation content and its result analysis:

[0089] In the simulation experiment of the present invention, the present invention and four existing technologies (conditional autoencoder C2AE open-set recognition method, variance consistency constraint prototype SLCPL open-set recognition method, prototype GCPL open-set recognition method, and convolutional neural network OpenMax open-set recognition method) are respectively used to perform open-set recognition on the input vehicle SAR image targets. In the simulation experiment, the two existing technologies adopted refer to:

[0090] The prior art self-encoding C2AE open-set recognition method refers to the optical image open-set recognition method proposed by Poojan Oza et al. in "C2AE: Class-Conditioned Auto-Encoder for Open-set Recognition, IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019", abbreviated as the C2AE open-set recognition method.

[0091] The prior art variance-consistent prototype SLCPL open-set recognition method refers to the optical image open-set recognition method proposed by Xia et al. in "Spatial Location Constraint Prototype Loss for Open Set Recognition, arXiv preprint arXiv:2110.11013. 2021", abbreviated as the SLCPL open-set recognition method.

[0092] The prior art prototype GCPL open-set recognition method refers to the optical image open-set recognition method proposed by Yang et al. in "Robust classification with convolutional prototype learning, Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2018.", abbreviated as the GCPL open-set recognition method.

[0093] The prior art convolutional neural network OpenMax open-set recognition method refers to the optical image open-set recognition method proposed by Bendale A et al. in "Towards open set deep networks, IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016", abbreviated as the OpenMax open-set recognition method.

[0094] The simulation experiment of the present invention randomly selects K classes and corresponding class labels from the training sample set to form a training set (the value range of K is [3, 7]). The test set is composed of all ten classes of vehicle images and class labels with the radar working elevation angle of 15°. The open-set recognition simulation experiment is repeated 5 times at each opening degree.

[0095] To ensure fairness, the present invention and the comparative method are carried out under the same experimental data division.

[0096] The open-set recognition results of the present invention and the comparative method under different openness are evaluated using an evaluation metric (F1 score). Using the following formula, the F1 score is calculated, and all the calculation results are plotted as Figure 3 :

[0097]

[0098] where TP i represents the number of images of the i-th class that are correctly recognized, FN i represents the number of images of the i-th class that are misjudged as rejected, FP i represents the number of images of the unknown class that are predicted as the i-th class, and TN i represents the number of images of the unknown class that are correctly rejected.

[0099] Refer to Figure 3 for a further description of the simulation experiment results of the present invention.

[0100] Figure 3 In , C2AE, SLCPL, GCPL, and OpenMax respectively represent four different open-set recognition methods in the prior art, and the F1 score of the open-set performance metrics for randomly repeated experiments multiple times under different openness. Figure 3 In , the current method broken line represents the open-set recognition technology proposed by the present invention, and the F1 score of the open-set performance metrics for randomly repeated experiments multiple times under different openness. The present invention can maintain a high F1 score index under different openness. In the case of small openness, the Proposed technology has increased by 5 percentage points compared to the prior art; in the case of large openness, the present invention still maintains a certain advantage compared to the prior art. It can be seen from Figure 3 that the proposed SAR image target open-set recognition technology combining hard-to-separate sample generation and learning of the present invention is more robust for real open scenarios.

[0101] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various modifications and changes in form and details may be made without departing from the principle and structure of the present invention. However, these corrections and changes based on the idea of the present invention are still within the protection scope of the claims of the present invention.

Claims

1. A method for open-set recognition of SAR image targets combining hard sample generation and learning, characterized in that, Training and constructing a convolutional neural network for outputting the target prediction probability distribution of SAR images through the generation and learning of hard-to-separate samples; the steps of this recognition method are as follows: Step 1, generate the training set: Each of the 2746 images with a radar working pitch angle of 17° in the sample set is cropped into an image of 64×64 pixels. All the selected images contain 10 categories. Within the range of class labels [1, 10], the target category of each image is labeled. Then, all the cropped images and the corresponding class labels are combined to form the training set; Step 2, construct an open-set recognition convolutional neural network: Step 2.1, construct the generator sub-network: Build a generator sub-network composed of five deconvolutional layers in series; set the number of convolutional kernels of the first to fourth deconvolutional layers to 1024, 512, 256, 128, and 3 in sequence, the size of the convolutional kernels to 4×4, the stride of the convolutional kernels to 1, 2, 2, 2, and 2 in sequence, the padding method to the same-size padding method, and the bias setting to 0; Step 2.2, construct the discriminator sub-network: Build a generator sub-network composed of four convolutional layers in series; set the number of convolutional kernels of the first to fourth convolutional layers to 128, 256, 512, and 1024 in sequence, the size of the convolutional kernels to 4×4, the stride of the convolutional kernels to 1, 1, 1, and 0 in sequence, the padding method to the same-size padding, and the bias setting to 0; Step 2.3, construct the classifier sub-network: Construct a classifier sub-network composed of a group of convolutional layers and a fully connected layer in series. Among them, a group of convolutional layers is composed of nine convolutional layers in series; set the number of convolutional kernels of the first to ninth convolutional layers to 64, 64, 128, 128, 128, 128, 128, 128, and 128 in sequence, the size of the convolutional kernels to 3×3, the stride of the convolutional kernels to 1, 1, 2, 1, 1, 2, 1, 1, and 2 in sequence, the padding method to the same-size padding, and the bias setting to 0; set the parameters of the fully connected layer to 128×K, where K represents the total number of training sample categories, and the bias setting to 0; Step 2.4, connect the generator sub-network and the discriminator sub-network in parallel and then connect them in series with the classifier sub-network to form an open-set recognition convolutional neural network; Step 3, train the open-set recognition convolutional neural network in combination with hard-to-separate samples: Step 3.1, input the noise sequence randomly sampled from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated image output by the generator sub-network and the discriminative confidence score of each noise-generated image output by the discriminator sub-network are obtained; input each image in the training set into the open-set recognition convolutional neural network, and the discriminative confidence score of each image output by the discriminator sub-network is obtained; use the cross-entropy loss function to calculate the loss value between the discriminative confidence score of each image and the domain label corresponding to the image to obtain the total loss function of the discriminator sub-network; use the Adam optimization algorithm to update the current discriminator sub-network parameters; Step 3.2, Input the noise sequence randomly sampled again from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated images output after passing through the generator subnet are input into the discriminator subnet to output the discrimination confidence scores for each noise-generated image, and then input into the classifier subnet to output the predicted probability distribution for each noise-generated image. Using the cross-entropy loss function, calculate the loss value between the predicted probability distribution of each noise-generated image and the class label corresponding to this image, and calculate the loss value between the discrimination confidence score of each noise-generated image and the domain label, to obtain the total loss function of the generator subnet. Use the Adam optimization algorithm to update the current generator subnet parameters; Step 3.3, Input each image in the training set into the open-set recognition convolutional neural network. After passing through the classifier subnet, output the known target class features for each image, and calculate the hard-to-separate sample features corresponding to each image using the hard-to-separate sample feature generation function based on gradient descent; Step 3.4, Input the noise sequence randomly sampled again from the standard normal distribution into the open-set recognition convolutional neural network. The noise-generated images output after passing through the generator subnet are input into the discriminator subnet, and hard-to-separate images with higher discrimination confidence scores are generated according to the discrimination confidence scores. After passing through the classification subnet, output the hard-to-separate image features for each hard-to-separate image. The hard-to-separate image features and the hard-to-separate sample features together form the hard-to-separate feature set, and the known target class features form the training feature set; Step 3.5, Input the training feature set and the hard-to-separate feature set into the open-set recognition convolutional neural network. After passing through the classifier subnet, output the predicted probability distribution for each feature. Using the cross-entropy loss function, calculate three loss functions respectively. Calculate the loss value between each feature in the training feature set and the class label corresponding to this feature, calculate the loss value between each feature in the training feature set and the smoothed label corresponding to this feature, and calculate the loss value between each feature in the hard-to-separate feature set and the uniform distribution, to obtain the total loss function of the classifier subnet. Use the SGDM optimization algorithm to update the current classifier subnet parameters; Step 3.6, Repeat Steps 3.1 to 3.5, and iteratively update the parameters of the generator subnet, discriminator subnet, and classifier subnet in the open-set recognition convolutional neural network for the number of training iterations, to obtain the trained open-set recognition convolutional neural network; Step 4, Set the rejection threshold of the open-set recognition convolutional neural network; Step 5, Obtain the predicted probability distribution of the SAR image target to be recognized; In the same way as in Step 1, crop the SAR image to be recognized to obtain the cropped SAR image target, and input the cropped SAR image target into the open-set recognition convolutional neural network. After passing through the classifier subnet, output the corresponding predicted probability distribution; Step 6, Determine whether the maximum value in the predicted probability distribution of the SAR image target to be recognized is greater than the rejection threshold. If so, execute Step 7; otherwise, execute Step 8; Step 7, Select the class corresponding to the maximum score in the predicted probability distribution of the SAR image target as the recognition result and output it; Step 8, determine the SAR image target as an unknown target class and output it.

2. The SAR image target open-set recognition method for generating and learning in combination with inseparable samples according to claim 1, wherein The total loss function of the discriminator sub-network described in Step 3.1 is as follows: Among them, L D represents the total loss function of the discriminator sub-network, N0 represents the total number of training set images and the total number of noise sequences, Σ represents the summation operation, i represents the image serial number of the training set images, j represents the noise serial number of the noise sequences, log represents the logarithm operation with base 2, D(x (i) ) represents the discriminative confidence score output after inputting the i-th image in the training set images into the open-set recognition convolutional neural network and passing through the discriminator sub-network, D(G(z (j) )) represents the noise-generated image output after inputting the j-th noise in the noise sequences into the open-set recognition convolutional neural network, passing through the generator sub-network, and then passing through the discriminator sub-network, and the discriminative confidence score output after that.

3. The SAR image target open set recognition method for generating and learning in combination with difficult-to-separate samples according to claim 2, characterized in that The total loss function of the generator sub-network described in Step 3.2 is as follows: Among them, L G represents the total loss function of the generator sub-network, λ represents the training hyperparameter, K represents the total number of categories of the training set images, C(G(z (j) )) represents the noise-generated image output after the j-th noise in the noise sequence is input into the open-set recognition convolutional neural network and passed through the generator sub-network, and the predicted probability distribution output after passing through the generator sub-network.

4. The SAR image target open set recognition method for generating and learning in combination with difficult-to-separate samples according to claim 3, wherein The difficult sample feature generation function described in Step 3.3 is as follows: Among them, L U represents the loss function for generating hard-to-separate sample features, represents the class smoothed label corresponding to the i-th image in the training set images, α represents a parameter, represents a uniform distribution vector, f (i) represents inputting the i-th image in the training set images into the open-set recognition convolutional neural network, and outputting the corresponding known target class features after passing through the classifier sub-network, represents obtaining the corresponding hard-to-separate sample features after passing the i-th image in the training set images through the hard-to-separate sample feature generation function of gradient descent.

5. The SAR image target open set recognition method for combining and learning difficult-to-separate samples according to claim 4, characterized in that, The total loss function of the classifier sub-network described in Step 3.5 is as follows: Among them, L C represents the total loss function of the classifier sub-network, k represents the serial number of the features in the training feature set, N1 represents the total number of difficult-to-separate feature sets, and l represents the serial number of the features in the difficult-to-separate feature set.

6. The SAR image target open set recognition method for combining and learning difficult-to-separate samples according to claim 5, wherein The formula for setting the rejection threshold described in Step 4 is as follows: τ = (1 - α) + α / K where τ represents the rejection threshold.

Images