Unseen category target sar image recognition method, device, equipment and medium

Abstract

The application relates to a kind of unseen category target SAR image recognition method, device, equipment and medium.The method comprises: by extracting corresponding features from measured SAR image, visible category simulation image and unseen category simulation image in training sample set, training feature generation network using measured feature and visible category feature, obtain the feature generation network with the ability of mapping category feature to false measured feature, obtain the unseen category false measured feature of unseen category feature using feature generation network, inverse mapping is carried out on the unseen category false measured feature using category feature reconstruction unit, obtain false unseen category feature and intermediate layer embedding feature, after splicing unseen category false measured feature, false unseen category feature and intermediate layer embedding feature, training classifier is carried out, obtain the trained classifier, and the SAR measured image of unseen category target is recognized using the classifier.

Description

Technical Field

[0001] This application relates to the field of automatic SAR image recognition technology, and in particular to a method, apparatus, device and medium for recognizing SAR images of unseen targets. Background Technology

[0002] Synthetic Aperture Radar (SAR) is an active remote sensing technology that uses radar principles to acquire information about ground features. Unlike optical remote sensing, SAR uses microwave signals to observe the Earth's surface, offering all-weather, all-time coverage and high resolution. Therefore, SAR images are widely used in various fields. A standard SAR ATR system consists of three main parts: detection, identification, and classification. The final classification task aims to classify and identify the identified targets. Due to clutter and interference, and the unclear and discontinuous outlines of targets, SAR target classification has long been a challenging research problem.

[0003] SAR target classification methods can be divided into traditional methods and deep learning-based methods. Traditional methods include template matching, model-based methods, and machine learning-based methods. Among them, machine learning-based methods are fast, have high recognition capabilities, and good robustness. However, machine learning-based methods rely on manually designed feature extractors, which requires professional knowledge and experience. Compared to traditional methods, deep learning-based methods can automatically extract features and have stronger generalization ability and recognition performance. Deep learning-based methods have also shown good performance in the field of SAR image recognition and are gradually becoming a new research hotspot in SAR image classification tasks.

[0004] However, deep learning-based methods are data-driven and typically require large amounts of data for training to extract effective features. When data is scarce, the network's recognition ability drops significantly. SAR images, due to limitations in technology, resources, environment, and policy, make it difficult to obtain large amounts of data; therefore, the few-shot problem has become a prominent issue in SAR image target recognition. To address the few-shot problem in SAR image classification, researchers have proposed various solutions. For example, expanding the training dataset is a common approach, while methods based on transfer learning and meta-learning are the main approaches. Although few-shot recognition methods can address the data scarcity problem to some extent, they cannot identify unseen category targets where training data is completely missing. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, device, and medium for identifying unseen target categories in SAR images, which can identify unseen target categories, in order to address the aforementioned technical problems.

[0006] A method for identifying unseen target categories in SAR images, the method comprising:

[0007] A training sample set is obtained, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0008] For the visible category sample set, feature extraction is performed on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and the visible category features, respectively;

[0009] The feature generation network is trained using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features. The category generation network includes a category feature reconstruction unit.

[0010] Feature extraction is performed on the unseen category simulation image to obtain unseen category features. The trained feature generation network is then used to map the unseen category features to obtain false measured features of the unseen category.

[0011] The category feature reconstruction unit is used to perform inverse mapping on the false measured features of the unseen category to obtain false unseen category features and intermediate layer embedding features. The false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain a trained classifier.

[0012] Acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0013] In one embodiment, a pre-trained feature extraction network is used to extract features from the measured SAR image, the visible category simulation image, the unseen category simulation image, and the unseen category measured SAR image. The feature extraction network is a fully convolutional network.

[0014] The classification and recognition network includes a trained feature extraction network, a category feature reconstruction unit, and a trained classifier connected in sequence.

[0015] In one embodiment, when the feature extraction network is used to obtain the visible category features from the visible category simulation image:

[0016] In the visible category sample set, multiple visible category simulation features are obtained by passing the feature extraction network to all visible category simulation images applying the same target category;

[0017] The average of multiple visible category simulation features is used to obtain the category feature corresponding to the visible category.

[0018] In one embodiment, the feature generation network includes a conditional generation network, a category feature reconstruction unit, and a feedback unit;

[0019] The conditional generative network is obtained by fusing a conditional variational autoencoder and a conditional generative adversarial network, wherein the conditional variational autoencoder and the conditional generative adversarial network share a generator.

[0020] In one embodiment, in the feature generation network:

[0021] The conditional variational autoencoder uses the category features as conditions to map the measured features to a low-dimensional vector in the latent space, and then re-decodes the low-dimensional vector to obtain false measured features.

[0022] The conditional generative adversarial network takes the category features as conditions, the measured features and training fake measured features as inputs, and outputs the degree of truth or falsehood of the input features.

[0023] The category feature reconstruction unit performs inverse mapping on the false measured features to obtain false category features and intermediate embedded features, and feeds the intermediate embedded features back to the shared generator in the conditional generation network through the feedback unit.

[0024] In one embodiment, during the training of the feature generation network:

[0025] The conditional generation network is optimized using the KL divergence loss function, the reconstruction loss function calculated based on the false measured features and the measured features, and the adversarial loss function calculated based on the degree of truth or falsehood of the input features.

[0026] The category feature reconstruction unit is optimized based on the cycle consistency loss function calculated from the false category features and the visible category features, and the conditional generation network is optimized again.

[0027] In one embodiment, a cross-entropy loss function is used when training the classifier.

[0028] This application also provides a device for identifying unseen target categories in SAR images, the device comprising:

[0029] The training sample set acquisition module is used to acquire a training sample set, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0030] The visible category feature extraction module is used to extract features from the measured SAR images and the simulated images of the visible category corresponding to the same target category in the visible category sample set, so as to obtain the measured features and the visible category features respectively.

[0031] The feature generation network training module is used to train the feature generation network using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features, wherein the category generation network includes a category feature reconstruction unit.

[0032] The module for obtaining false measured features of unseen categories is used to extract features from the simulation image of the unseen category to obtain unseen category features, and to map the unseen category features using the trained feature generation network to obtain false measured features of the unseen category.

[0033] The classifier training module is used to perform inverse mapping on the false measured features of the unseen category using the category feature reconstruction unit to obtain false unseen category features and intermediate layer embedding features. The classifier is then trained by concatenating the false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features to obtain a trained classifier.

[0034] The unseen category target recognition module is used to acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0035] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0036] A training sample set is obtained, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0037] For the visible category sample set, feature extraction is performed on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and the visible category features, respectively;

[0038] The feature generation network is trained using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features. The category generation network includes a category feature reconstruction unit.

[0039] Feature extraction is performed on the unseen category simulation image to obtain unseen category features. The trained feature generation network is then used to map the unseen category features to obtain false measured features of the unseen category.

[0040] The category feature reconstruction unit is used to perform inverse mapping on the false measured features of the unseen category to obtain false unseen category features and intermediate layer embedding features. The false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain a trained classifier.

[0041] Acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0042] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0043] A training sample set is obtained, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0044] For the visible category sample set, feature extraction is performed on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and the visible category features, respectively;

[0045] The feature generation network is trained using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features. The category generation network includes a category feature reconstruction unit.

[0046] Feature extraction is performed on the unseen category simulation image to obtain unseen category features. The trained feature generation network is then used to map the unseen category features to obtain false measured features of the unseen category.

[0047] The category feature reconstruction unit is used to perform inverse mapping on the false measured features of the unseen category to obtain false unseen category features and intermediate layer embedding features. The false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain a trained classifier.

[0048] Acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0049] The aforementioned method, apparatus, device, and medium for recognizing SAR images of unseen targets extract features from measured SAR images and simulated images of the same target category in the visible category sample set of the training sample set. These extracted features yield measured features and visible category features, respectively. A feature generation network is trained using these two features, enabling it to map category features to false measured features. The trained network then maps unseen category features to false measured features. A category feature reconstruction unit performs inverse mapping on these false measured features to obtain false unseen category features and intermediate layer embedded features. These false measured features, false unseen category features, and intermediate layer embedded features are concatenated and used to train a classifier. Finally, features are extracted from measured SAR images of unseen targets, and the extracted features are input into a classification and recognition network including the trained classifier to obtain the target recognition result for the measured SAR images of unseen targets. This method can recognize measured SAR images of unseen targets. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for identifying SAR images of unseen targets in one embodiment.

[0051] Figure 2 This is a schematic diagram of the overall framework for training and testing the method described in this paper in one embodiment;

[0052] Figure 3 This is a schematic diagram of the feature extraction network structure in one embodiment;

[0053] Figure 4This is a schematic diagram of the category feature extraction process in one embodiment;

[0054] Figure 5 This is a schematic diagram of the feature generation network structure in one embodiment;

[0055] Figure 6 This is a schematic diagram of the classifier structure in one embodiment;

[0056] Figure 7 This is a schematic diagram of samples in the dataset during a simulation experiment in one embodiment;

[0057] Figure 8 This is a structural block diagram of a SAR image recognition device for unseen target categories in one embodiment;

[0058] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] In existing technologies, when using deep learning models for target recognition in SAR images, target recognition relies on a large number of labeled samples and cannot classify unseen target categories. Due to the lack of effective category-aided information, existing zero-shot SAR image target recognition methods are typically limited to inferring one type of unseen target category and cannot classify multiple types of unseen target categories. In this application, as... Figure 1 As shown, a method for identifying unseen target categories in SAR images is provided, including the following steps:

[0061] Step S100: Obtain a training sample set, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, while the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0062] Step S110: For the visible category sample set, perform feature extraction on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and visible category features respectively.

[0063] Step S120: The feature generation network is trained using measured features and visible category features, so that the trained feature generation network has the ability to map category features to fake measured features. The category generation network includes a category feature reconstruction unit.

[0064] Step S130: Extract features from the unseen category simulation image to obtain unseen category features, and use the trained feature generation network to map the unseen category features to obtain false measured features of the unseen category.

[0065] Step S140: The class feature reconstruction unit is used to reverse map the false measured features of the unseen class to obtain false unseen class features and intermediate layer embedding features. The false measured features of the unseen class, the false unseen class features and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain the trained classifier.

[0066] Step S150: Obtain the unseen category measured SAR image, extract features from the unseen category measured SAR image, input the extracted features into the classification and recognition network, and obtain the target recognition result of the unseen category measured SAR image. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0067] In this embodiment, deep features are first extracted from the simulated image and further fused into category features, which can represent the entire category. Then, a conditional VAE-GAN network (i.e., a feature generation network) is constructed to generate feature instances for unseen categories. The category features of both visible and unseen categories contain shared high-level semantic information, which helps to generalize the mapping learned from the visible categories to the unseen categories. Finally, a classifier is trained using the generated features of the unseen categories, which can classify test images of the unseen categories. This method can be generalized to generalized zero-shot recognition tasks.

[0068] In real-world SAR target recognition scenarios, it's impossible to obtain SAR images of non-cooperative targets in their actual environment beforehand for network training. Furthermore, certain real-world applications based on SAR images are often in open environments, requiring the ability to identify targets appearing for the first time for strike designation. Therefore, methods for identifying unseen targets are essential. The problem of identifying unseen categories with absolutely no training data is the zero-shot identification problem, an extreme case of the few-shot identification problem.

[0069] When dealing with zero-shot identification problems, existing neural network training methods often suffer from severe domain shift issues and discrete feature distributions due to the difficulty in guaranteeing the quality of semantic information and the lack of supervised learning. Embedding models can only infer one type of unseen category and cannot identify multiple unseen categories.

[0070] Specifically, to generate data on unseen target categories for supervised learning to achieve classification of multiple unseen target categories, this method proposes a zero-shot SAR target recognition approach based on a generative network conditioned on category features constructed from simulated images. First, features are extracted from both simulated and measured images, and the extracted simulated features are further fused into category features. Then, a conditional generative network is constructed. The network uses the category features of visible categories as conditions to learn the mapping from category features to measured features. The learned mapping can generalize to the generation of unseen category features. Finally, a classifier is trained using the generated features of unseen categories, enabling it to classify measured images of unseen categories.

[0071] like Figure 2 The diagram shows the overall architecture of the proposed method, which is divided into a training section and a testing section. The testing section is essentially the process of applying the method to identify unseen category targets. The training section includes feature extraction, feature generation, and classification. The technical features of steps S110 and S130 can be found in the feature extraction section. Step S120 uses the visible category training set to train the feature generation network, and in step S130, the trained feature generation network generates unseen category features based on the unseen category simulation image; these steps can be found in the feature generation section. Step S140 can be found in the classification section.

[0072] In step S100, it is assumed that there are a total of k target categories in the training sample set, where m target categories are considered visible category samples and n target categories are considered unseen category samples. The visible category sample set can be represented as S = {x} s ,y s ,a s |x s ∈X s ,y s ∈Y s ,a s ∈A s}, where x s For samples of m visible categories, For labels of the visible category m, This provides auxiliary information for the m visible categories. The sample set of unseen categories can be represented as U = {y}. u ,a u |y u ∈Y u ,a u ∈A u},in For labels of category n that have no known category, This provides auxiliary information for n categories where no category has been found. m + n = k.

[0073] Furthermore, the visible category samples include both measured and simulated SAR images of m types of targets, used to train the feature generation network to learn the mapping from category features to false measured features. Similarly, the n unseen category samples include both measured and simulated SAR images, with the classifier trained using simulated images of the n unseen categories. During the testing phase, measured SAR images of the unseen categories are used for testing.

[0074] In steps S110 and S130, and in the feature extraction of unseen category measured SAR images, a feature extraction network is used. In this embodiment, the feature extraction network is a pre-trained fully convolutional network. The fully convolutional network (A-ConvNet) uses only sparsely connected convolutional structures instead of fully connected networks, which greatly reduces the number of network parameters and achieves good performance in SAR vehicle target recognition tasks.

[0075] In this embodiment, CANet is improved upon A-ConvNet by designing it as the feature extraction network. This ensures that the extracted features are well-represented while also meeting the requirements for subsequent training. The specific structure of CANet is as follows: Figure 3 As shown.

[0076] Specifically, the CANet network consists of six layers. The first three layers are a combination of convolutional modules, ReLU activation functions, and pooling modules. The fourth layer is followed by a Dropout operation. The first four layers are designed based on A_ConvNet, with the addition of two separate convolutional layers at the end for extracting effective features. The original image size is 128×128×3, which is then resized to 88×88×3 before being input into the network. After passing through the fourth layer, the feature size is 3×3×128. The fifth convolutional kernel size is 3×3×256, enabling it to output features of size 1×1×256. The 256-dimensional feature dimension ensures the feature's ability to represent SAR image targets while reducing the complexity of subsequent model training. The last convolutional kernel is designed to be 1×1×n, enabling it to output probabilities for n classes for final loss calculation and gradient backpropagation.

[0077] The CANet network is used to extract measured features from real-world images. Since images of unseen categories cannot be used for training, the network is pre-trained using only images of visible categories. The extracted measured features of visible categories are then fed into the next stage for training.

[0078] In this embodiment, feature extraction of simulated images can also be performed using the aforementioned feature extraction network. Existing methods for generating simulated SAR images from actual data primarily focus on a one-to-one mapping from image input to output, which limits the generation of false measured data. However, this method utilizes a conditional generation network to achieve one-to-many generation; therefore, it uses simulated features to construct category features as conditions for generating multiple false measured features.

[0079] In this embodiment, when using a feature extraction network to obtain visible category features from visible category simulation images: the visible category sample set is used to obtain multiple visible category simulation features corresponding to all visible category simulation images applying the same target category through the feature extraction network; the multiple visible category simulation features are weighted and summed to obtain the category feature corresponding to the visible category.

[0080] Specifically, simulated images for each category are used to extract category features, and the extraction process is shown in Figure 4. First, CANet is pre-trained using real-world images of the visible categories. Then, for all simulated images in a category, the pre-trained network is used to extract the simulated feature 'a'. ij Furthermore, simulation characteristic features are fused into simulation domain category features. Since a category feature should reflect the most significant features of that category as much as possible, the category feature is defined by the average of all simulation features of that category, as shown in the following formula:

[0081]

[0082] In formula (1), k i This represents the number of simulated images for the i-th category. First, category features are extracted from the simulated images of the visible categories, and then fed into the next stage of training. Second, category features for the unseen categories are extracted from the simulated images of the unseen categories to generate false measured features for the unseen categories.

[0083] Feature generation learning is the second stage of the proposed framework. It uses the category features of visible categories and measured features to learn the mapping relationship between them, so that the final mapping can be better generalized to the generation of unseen categories.

[0084] In this embodiment, the feature generation network includes a conditional generation network, a category feature reconstruction unit, and a feedback unit. The conditional generation network is obtained by fusing a conditional variational autoencoder and a conditional generative adversarial network, and the conditional variational autoencoder and the conditional generative adversarial network have a shared generator.

[0085] Specifically, in this method, a conditional generative network is constructed by fusing VAE and GAN networks, and a category feature reconstruction unit and a feedback unit are introduced to improve the network's generative ability. Its structure is as follows: Figure 5As shown.

[0086] Furthermore, in the feature generation network: the conditional variational autoencoder uses category features as conditions to map the measured features to a low-dimensional vector in the latent space, and then re-decodes the low-dimensional vector to obtain training fake measured features; the conditional generative adversarial network uses category features as conditions, takes the measured features and training fake measured features as inputs, and outputs the degree of truth or falsehood of the input features; the category feature reconstruction unit performs inverse mapping on the training fake measured features to obtain fake category features and intermediate embedding features, and feeds the intermediate embedding features back to the shared generator in the conditional generative network through the feedback unit.

[0087] like Figure 5 As shown, x is the measured feature, and a is the class feature. The encoder E(x,a) and the shared generator G(z,a) constitute a conditional variational autoencoder (CVAE) with a condition. The shared generator G(z,a) and discriminator... A Conditional Generative Adversarial Network (CGAN) is constructed using 'a' as the condition. The Conditional Variational Autoencoder and the Conditional Generative Adversarial Network share the generator G(z,a), which is combined to form a Conditional VAE-GAN. VAE-GAN combines the advantages of VAE encoding the latent space and GAN generating high-quality features, allowing for more stable training and generating spurious tested features that smoothly transition to semantic features.

[0088] For the CVAE feature generation part, the class feature 'a' is concatenated with the measured feature 'x' as input to E(x,a). First, the encoder E(x,a) encodes the measured feature into a low-dimensional vector z in a continuous latent space. z is assumed to follow an isotropic Gaussian distribution; therefore, the output of E(x,a) is the mean and variance vector of a Gaussian distribution, i.e., (μ... x ,σ x Then, the generator G(z,a) re-decodes z in the latent space into spurious measured features.

[0089] In this embodiment, when training the conditional generation network, the size is reduced. The difference between x and y ensures the cyclic consistency of the generated features with the original features, optimized by the following loss function:

[0090]

[0091] In formula (2), the conditional distribution q(z|x,a) is the probability distribution modeled by E(x,a), and p(z|a) is the prior distribution that follows N(0,1). It is equal to G(z,a). KL It is the KL divergence between q(z|x,a) and p(z|a). GThis is the generation and reconstruction loss of CVAE, which is set here as the cross-entropy loss between the generated features and the original measured features.

[0092] For the feature generation part of CGAN, the generator G(z,a) is the same as the generator of CVAE. Using 'a' as a condition, and real measured features x and spurious measured features... As input, the output is a real number representing the degree of truth or falsehood of the input feature.

[0093] In this embodiment, the loss of GAN is optimized using an improved WGAN loss:

[0094]

[0095] In formula (3), The fake measured features generated by the producer. , where a ~ U(0,1), and λ is the coefficient of the penalty term. The first two terms approximate the Wasserstein distance between the distributions of false and true measured features, and the last term is the gradient penalty term, which forces the gradient at any point to approach the unit norm. The improved WGAN loss makes training smoother and more stable, alleviating the model collapse problem during training of traditional GAN ​​networks.

[0096] Furthermore, combining the losses from CVAE and CGAN, the overall optimization objective of Conditional VAE-GAN (Conditional Generative Network) is:

[0097]

[0098] In formula (4), α is a hyperparameter.

[0099] Furthermore, the feature generation network, based on the conditional generation network, introduces a category feature reconstruction unit (CFR) and a feedback module (F) to further enhance the network's generative capabilities. The CFR reconstructs the generated spurious measured features into category features, i.e., it learns the inverse mapping from measured features to categories to ensure the semantic consistency of the generated features.

[0100] In this embodiment, the feature generation network is optimized using a cycle consistency loss function:

[0101]

[0102] In formula (5), F feeds the CFR intermediate layer embedded feature h into G(z,a), which enables G(z,a) to iteratively improve feature generation, thereby obtaining an enhanced feature representation.

[0103] In this embodiment, the overall loss function of the entire feature generation network is as follows:

[0104] l Total =l VAE-GAN +βl R (6)

[0105] In formula (6), β is a hyperparameter.

[0106] After training a feature generation network that maps simulated category features to fake measured features using visible category measured features and category features, the third stage, classifier training, is performed. The goal is to train the classifier using fake measured features of unseen categories, enabling it to be used for classification and recognition of unseen category measured images. The classifier structure is as follows: Figure 6 As shown.

[0107] In this embodiment, the classifier retains the category feature reconstruction unit instead of directly utilizing the false measured features of unseen categories. Train the classifier. The CFR is the inverse mapping of G(z,a), where the intermediate layer embeds features h and the reconstructed class features are used. It encodes category information that is complementary to the generated feature instances, which can be used as an auxiliary information source to assist in training the classifier.

[0108] In this embodiment, The classifier is trained by concatenating h as a classification feature c. The classifier outputs the probability of no class being seen. Simultaneously, cross-entropy loss l is applied. C optimization:

[0109]

[0110] In formula (7), θ is the parameter of the classifier. This represents the probability of the output category.

[0111] After the classifier is trained, the entire training phase is complete. Then, in the testing phase, real-world SAR images of unseen categories are used for testing.

[0112] In this embodiment, during testing and subsequent actual recognition, a classification and recognition network is constructed using a trained feature extraction network, a category feature reconstruction unit, and a trained classifier. The network structure can be referenced from [reference needed]. Figure 2 The network structure in the test section allows target classification to be achieved by inputting measured SAR images of unknown categories into the classification and recognition network.

[0113] In another embodiment, reference Figure 2 The entire training process includes:

[0114] In the feature extraction stage, the measured features x of the visible category are extracted using the feature extraction module.s Simultaneously, category features for both visible and unseen categories are extracted. and Where, x s , and Both are 256-dimensional features.

[0115] During the feature generation training phase, x... s With the corresponding category The concatenated values ​​are fed into the encoder E(x,a), and the 256-dimensional mean and variance of the latent code z are output separately. Then, the latent code z is concatenated with... The concatenated data is fed into G(z,a), and the output is a reconstructed, spurious, measured characteristic feature. G(z,a) consists of two fully connected layers. The first layer has 2048 neurons, and the second layer has 256 neurons, which is equal to the measured feature dimension. Then, and Splicing to train the discriminator E,G,D through loss l VAE-GAN Preliminary optimization.

[0116] The data is further fed into the CFR module for reconstructing categorical features. The CFR consists of two fully connected layers. The first layer has 2048 neurons, and the second layer has the same number of neurons as the categorical features. The 2048-dimensional vector output from the first layer of the CFR is called the intermediate layer feature h, which is fed into the feedback layer F(h). F(h) is also a two-layer fully connected layer, with both layers having 2048 neurons, the same number as the input intermediate layer feature h. h is then weighted and fused with the features from the first fully connected layer of the producer G(z,a) after passing through the feedback layer F(h). The CFR and F are then fused through a loss function l. R Optimization, l R Further constrain the learning parameters E, G, and D.

[0117] In the classification training phase, firstly, random noise is used with... splicing generates false measured features of unseen categories. Next, Input F(h), and output spurious class features The intermediate layer features h are concatenated and fed into the final classification network C to train the classifier. The classifier parameters are obtained through the loss function l. C optimization.

[0118] During the testing phase, pre-trained CFRs were used to extract measured feature x from unseen category measured images. u The data is then input into the trained classification module for classification and recognition.

[0119] In this embodiment, the training and testing processes are integrated into one, with a test performed after each training round.

[0120] In this paper, the effectiveness of the proposed method is also demonstrated through experiments.

[0121] The experiments used two datasets: the publicly available SAMPLE dataset and the MSTAR dataset. The SAMPLE dataset is a subset of the SAMPLE dataset, containing both measured and simulated SAR images of ten types of vehicle targets. The images are 128×128 pixels in size with a resolution of 0.3m×0.3m, and there is a one-to-one correspondence between the simulated and measured images. The elevation angle range is 14°–17°, and the azimuth angle range is 10°–80°. The azimuth angle does not fully cover the 10°–80° range, with some angles missing. The MSTAR dataset, developed with funding from the Defense Advanced Research Projects Agency (DARPA), contains ten types of vehicle targets with a resolution of 0.3m×0.3m. The elevation angle range includes 15°, 17°, 30°, and 45°, and the azimuth angle range is 0°–360°. This paper utilizes images with elevation angles of 16° and 17° from the SAMPLE dataset and images with an elevation angle of 15° from the MSTAR dataset for the experiments.

[0122] In the experiment, the SAMPLE and MSTAR datasets were combined, and two experimental groups were set up to correspond to the simulation and experimental categories: the SAMPLE-SAMPLE experimental group and the SAMPLE-MSTAR experimental group. The SAMPLE-SAMPLE experimental group contains ten target categories, where the simulation images and experimental images are the same as the images in the SAMPLE dataset. Since the simulation images in the SAMPLE dataset do not completely correspond to the experimental images in the MSTAR dataset, the SAMPLE-MSTAR dataset contains five target categories common to both datasets. The experimental images are the experimental images from the MSTAR dataset, and the simulation images are the simulation images from the SAMPLE dataset. Figure 7 Two pairs of simulated and measured images are shown for each category in the two experimental groups. The unseen categories in both experimental groups consist of three classes: BMP2, BTR70, and T72, used for testing; the visible categories consist of the remaining 7 and 2 classes of objects in the corresponding dataset, used for network training. The specific composition is shown in Table 1-2.

[0123] Table 1 SAMPLE-SAMPLE dataset

[0124]

[0125] Table 2 SAMPLE-MASTAR dataset

[0126]

[0127] Ten independent training tests were conducted in the two experimental groups. Tables 3 and 4 show the lowest, highest, and average recognition accuracy (with standard deviation) of the proposed method in the 10 training tests in the two experimental groups.

[0128] Table 3 Results of the SAMPLE-SAMPLE experiment

[0129]

[0130] Table 4. Results of the SAMPLE-MSTAR experiment

[0131]

[0132] The experiments directly trained classic networks A-ConvNet, ResNet18, and Vgg16 using simulated images and tested them on real-world images for comparison. The RN18+ method is a network designed for training and classification strategies on the SAMPLE dataset. Additionally, the results of training the feature extraction network CANet using only cross-entropy loss are presented, denoted as the Baseline. The Baseline is an improvement on A-ConvNet, and its final recognition rate is superior to A-ConvNet, indicating that the CANet network design in this paper ensures effective feature extraction. The proposed method achieves a recognition rate of 99.80% ± 0.22% and a minimum of 99.35%, both outperforming all compared methods. RN18+ and the Baseline achieve a maximum recognition rate of 100%, while the proposed method maintains the maximum 100% recognition rate while having the smallest standard deviation, demonstrating stable recognition performance. The focus is on the SMPLE-MSTAR experimental group, where the recognition rates of the compared methods range from 40% to 50%. The final recognition rate of the proposed method is 71.57% ± 2.28%, representing an improvement of nearly 20%-30% compared to classic methods.

[0133] The proposed method can identify three unseen target categories in two experimental groups. The results demonstrate the effectiveness of the proposed method. This is because the generated false measured features have a stronger similarity to the distribution of real measured features, thus the trained classification network exhibits stronger generalization ability than one trained directly using simulation data.

[0134] In the aforementioned SAR image recognition method for unseen target categories, category features are constructed using simulated images. These proposed category features are used to characterize the entire category, and their effectiveness and feasibility in assisting zero-shot learning have been verified. A zero-shot generation framework for SAR data based on conditional VAE-GAN is proposed. It learns the mapping from simulated information to measured data using only visible category data, enabling it to generalize to the generation of unseen category data. This method can classify multiple unseen SAR target categories, rather than being limited to inferring from a single unseen category.

[0135] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0136] In one embodiment, such as Figure 8 As shown, a SAR image recognition device for unseen target categories is provided, comprising: a training sample set acquisition module 200, a visible category feature extraction module 210, a feature generation network training module 220, an unseen category false measured feature acquisition module 230, a classifier training module 240, and an unseen target recognition module 250, wherein:

[0137] The training sample set acquisition module 200 is used to acquire a training sample set, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0138] The visible category feature extraction module 210 is used to extract features from the measured SAR images and the simulated images of the visible category corresponding to the same target category in the visible category sample set, so as to obtain the measured features and the visible category features respectively.

[0139] The feature generation network training module 220 is used to train the feature generation network using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features, wherein the category generation network includes a category feature reconstruction unit.

[0140] The module 230 for obtaining false measured features of unseen categories is used to extract features from the simulation image of the unseen category to obtain unseen category features, and to map the unseen category features using the trained feature generation network to obtain false measured features of unseen categories.

[0141] The classifier training module 240 is used to perform inverse mapping on the false measured features of the unseen category using the category feature reconstruction unit to obtain false unseen category features and intermediate layer embedding features. The classifier is trained by concatenating the false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features to obtain a trained classifier.

[0142] The unseen category target recognition module 250 is used to acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0143] Specific limitations regarding the SAR image recognition device for unseen targets can be found in the limitations of the SAR image recognition method for unseen targets mentioned above, and will not be repeated here. Each module in the aforementioned SAR image recognition device for unseen targets can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0144] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a SAR image recognition method for unseen target categories. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0145] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0146] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0147] A training sample set is obtained, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0148] For the visible category sample set, feature extraction is performed on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and the visible category features, respectively;

[0149] The feature generation network is trained using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features. The category generation network includes a category feature reconstruction unit.

[0150] Feature extraction is performed on the unseen category simulation image to obtain unseen category features. The trained feature generation network is then used to map the unseen category features to obtain false measured features of the unseen category.

[0151] The category feature reconstruction unit is used to perform inverse mapping on the false measured features of the unseen category to obtain false unseen category features and intermediate layer embedding features. The false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain a trained classifier.

[0152] Acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0154] A training sample set is obtained, which includes a visible category sample set and an unseen category sample set. The visible category sample set includes measured SAR images and simulated images of various visible category targets, and the unseen category sample set includes unseen category simulated images of various unseen category targets.

[0155] For the visible category sample set, feature extraction is performed on the measured SAR images and the simulated visible category images corresponding to the same target category to obtain the measured features and the visible category features, respectively;

[0156] The feature generation network is trained using the measured features and visible category features, so that the trained feature generation network has the ability to map category features to false measured features. The category generation network includes a category feature reconstruction unit.

[0157] Feature extraction is performed on the unseen category simulation image to obtain unseen category features. The trained feature generation network is then used to map the unseen category features to obtain false measured features of the unseen category.

[0158] The category feature reconstruction unit is used to perform inverse mapping on the false measured features of the unseen category to obtain false unseen category features and intermediate layer embedding features. The false measured features of the unseen category, the false unseen category features, and the intermediate layer embedding features are concatenated and then used to train the classifier to obtain a trained classifier.

[0159] Acquire unseen category measured SAR images, extract features from the unseen category measured SAR images, input the extracted features into a classification and recognition network, and obtain the target recognition result of the unseen category measured SAR images. The classification and recognition network includes a category feature reconstruction unit and a trained classifier.

[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for recognizing a never-classified target SAR image, characterized in that, The method comprises: obtaining a training sample set, the training sample set comprising a visible category sample set and an unseen category sample set, wherein the visible category sample set comprises measured SAR images corresponding to a plurality of visible category targets and visible category simulation images, and the unseen category sample set comprises unseen category simulation images of a plurality of unseen category targets; performing feature extraction on the measured SAR images and the visible category simulation images corresponding to the same target category in the visible category sample set to obtain measured features and visible category features respectively; training a feature generation network using the measured features and the visible category features, so that the trained feature generation network has the ability to map category features to false measured features, wherein the feature generation network comprises a conditional generation network, a category feature reconstruction unit and a feedback unit, the conditional generation network being obtained by fusing a conditional variational autoencoder and a conditional generative adversarial network, the conditional variational autoencoder and the conditional generative adversarial network having a shared generator, in the feature generation network: the conditional variational autoencoder maps the measured features to a low-dimensional vector in a latent space conditioned on the category features, and then decodes the low-dimensional vector to obtain false measured features, the conditional generative adversarial network takes the measured features and the training false measured features as inputs conditioned on the category features, and outputs the authenticity of the input features, the category feature reconstruction unit inversely maps the false measured features to obtain false category features and intermediate embedding features, and feeds the intermediate embedding features back to the shared generator in the conditional generation network through the feedback unit; performing feature extraction on the unseen category simulation images to obtain unseen category features, and mapping the unseen category features using the trained feature generation network to obtain unseen category false measured features; inversely mapping the unseen category false measured features using the category feature reconstruction unit to obtain false unseen category features and intermediate layer embedding features, and training a classifier after concatenating the unseen category false measured features, the false unseen category features and the intermediate layer embedding features to obtain a trained classifier; obtaining an unseen category measured SAR image, performing feature extraction on the unseen category measured SAR image, inputting the extracted features into a classification recognition network to obtain a target recognition result of the unseen category measured SAR image, wherein the classification recognition network comprises a category feature reconstruction unit and a trained classifier.

2. The method of claim 1, wherein the unseen class object SAR image recognition method is characterized by, The measured SAR images, the visible category simulation images, the unseen category simulation images and the unseen category measured SAR image are extracted using a pre-trained feature extraction network, and the feature extraction network uses a fully convolutional network; The classification recognition network comprises a trained feature extraction network, a category feature reconstruction unit and a trained classifier connected in sequence.

3. The method of claim 2, wherein the unseen class object SAR image recognition method is characterized by, When the feature extraction network is used to obtain the visible category features from the visible category simulation images: The visible category sample set is used to obtain a plurality of visible category simulation features corresponding to all visible category simulation images of the same target category through the feature extraction network. The plurality of visible category simulation features are averaged to obtain a category feature corresponding to the visible category.

4. The method of claim 3, wherein the unseen class object SAR image recognition method is characterized by, During the training of the feature generation network: The conditional generation network is optimized by using a KL divergence loss function, a reconstruction loss function calculated according to the false measured feature and the measured feature, and an adversarial loss function calculated according to the true and false degrees of the input feature; The category feature reconstruction unit is optimized according to a cycle consistency loss function calculated according to the false category feature and the visible category feature, and the conditional generation network is optimized again.

5. The method of claim 1, wherein the unseen class object SAR image recognition method is characterized by, A cross-entropy loss function is used during the training of the classifier.

6. An apparatus for recognizing a never-classified target SAR image, characterized by comprising: The device implements the unobserved category target SAR image recognition method of any one of claims 1-5, comprising: A training sample set acquisition module is configured to acquire a training sample set, wherein the training sample set includes a visible category sample set and an unobserved category sample set, the visible category sample set includes measured SAR images and visible category simulation images corresponding to a plurality of visible category targets, and the unobserved category sample set includes unobserved category simulation images of a plurality of unobserved category targets; A visible category feature extraction module is configured to perform feature extraction on the measured SAR images and the visible category simulation images corresponding to the same target category in the visible category sample set, and obtain measured features and visible category features, respectively; A feature generation network training module is configured to train a feature generation network by using the measured features and the visible category features, so that the trained feature generation network has the ability to map category features to false measured features, wherein the feature generation network includes a category feature reconstruction unit; An unobserved category false measured feature obtaining module is configured to perform feature extraction on the unobserved category simulation images to obtain unobserved category features, and map the unobserved category features by using the trained feature generation network to obtain unobserved category false measured features; A classifier training module is configured to inversely map the unobserved category false measured features by using the category feature reconstruction unit to obtain false unobserved category features and intermediate layer embedding features, splice the unobserved category false measured features, the false unobserved category features, and the intermediate layer embedding features, and train a classifier to obtain a trained classifier; An unobserved category target recognition module is configured to acquire an unobserved category measured SAR image, perform feature extraction on the unobserved category measured SAR image, input the extracted features into a classification recognition network, and obtain a target recognition result of the unobserved category measured SAR image, wherein the classification recognition network includes the category feature reconstruction unit and the trained classifier. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1-5. The processor executes the computer program to implement the steps of the method of any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which is executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

Images