A small sample target recognition method based on virtual feature enhancement and feature refinement

Abstract

The present invention discloses a small sample target recognition method based on virtual feature enhancement and feature refinement. First, a feature extractor is used to extract a low-dimensional feature map from a limited number of SAR training samples. Then, an embedded feature enhancer is used to calculate the similarity and search for features far from the class center, obtaining intra-class dissimilar feature pairs and inter-class similar feature pairs. The feature pairs are used to construct and enhance the corresponding virtual features. Then, a dynamic hierarchical feature refiner is used to dynamically generate convolution kernels to integrate local features with different dimensions into global features, thereby improving the intra-class compactness and inter-class separability of the features. Finally, a loss function combining adaptive loss and recognition loss is used to optimize the network, improve recognition accuracy, and complete target recognition. The method of the present invention solves the problems of difficult data acquisition and high annotation cost, improves the deep network learning effect and overall recognition accuracy, and has higher recognition accuracy and robustness than existing small sample target recognition methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition, and in particular relates to a small sample target recognition method based on virtual feature enhancement and feature refinement. Background Art

[0002] SAR is an important microwave remote sensing system, renowned for its ability to acquire high-resolution images around the clock and in all weather conditions. Automatic target recognition (ATR) is a widely researched and applied topic in the SAR field. However, ATR remains a key challenge in SAR applications. Therefore, the application of SAR ATR in civilian applications holds significant research significance.

[0003] Over the past few decades, a wide range of recognition algorithms and systems have emerged in the field of SAR ATR, primarily including template-based, model-based, and deep learning-based methods. While current deep learning-based SAR ATR methods demonstrate promising performance, they require sufficient training samples to supervise training, avoid overfitting, and achieve excellent generalization performance across diverse imaging scenarios. In practical applications, since most SAR image data is unlabeled or sparsely labeled, this can lead to overfitting in deep networks. In recent years, various methods have been proposed to mitigate the limited training sample problem in SAR ATR, such as augmentation methods based on transformation consistency, neighborhood, and generative adversarial networks. While these methods alleviate the small sample size issue to some extent, they do not substantially increase the amount of information used to supervise the training of recognition models. The paper "Z. Qin, H. Wang, et al., "Multi-instance attention network for few-shot learning," Information Sciences, vol. 611, pp. 464–475, 2022," suggests that the relative distribution information between sample features and class centers can effectively improve the features extracted by the recognition model, increasing the amount of information used to supervise the model. Therefore, it is crucial to study the relative distribution information of sample characteristics and class centers to improve the SAR ATR performance in small sample scenarios. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a small sample target recognition method based on virtual feature enhancement and feature refinement, which solves the problems caused by the limitation of labeled training samples and the fact that the total amount of supervised training information in existing small sample methods is not increased, thereby improving the performance of SAR ATR.

[0005] The technical solution of the present invention is a small sample target recognition method based on virtual feature enhancement and feature refinement, and the specific steps are as follows:

[0006] Step 1: In the training phase, a feature extractor is used to extract low-dimensional feature maps from limited SAR training samples;

[0007] Step 2: Using the embedded feature enhancer, we calculate the similarity and search for features far from the class center, obtain dissimilar feature pairs within the class and similar feature pairs between classes, and construct and enhance virtual features between features far from the class center.

[0008] Step 3: A dynamic hierarchical feature refiner is used to first enhance the class discriminability of local features. Then, a convolution kernel is dynamically generated based on the unique features of the input SAR image. The discriminative local features of different dimensions are integrated into the global features, improving the intra-class compactness and inter-class separability of features.

[0009] Step 4: Use a loss function that combines adaptive loss and recognition loss to optimize the network, improve recognition accuracy, and complete target recognition.

[0010] Beneficial effects of the present invention: The method of the present invention first uses a feature extractor to extract a low-dimensional feature map from a limited number of SAR training samples, then uses an embedded feature enhancer to calculate the similarity to search for features far away from the class center, obtains intra-class dissimilar feature pairs and inter-class similar feature pairs, uses the feature pairs to construct and enhance corresponding virtual features, and then uses a dynamic hierarchical feature refiner to dynamically generate convolution kernels to integrate local features with discriminability of different dimensions into global features, thereby improving the intra-class compactness and inter-class separability of features, and finally uses a loss function that combines adaptive loss and recognition loss to optimize the network, improve recognition accuracy, and complete target recognition. The method of the present invention solves the problems of difficult data acquisition and high labeling costs, improves the learning effect of deep networks and the overall recognition accuracy, and has higher recognition accuracy and robustness than existing small sample target recognition methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of a small sample target recognition method based on virtual feature enhancement and feature refinement of the present invention.

[0012] Figure 2 It is an overall framework diagram in an embodiment of the present invention.

[0013] Figure 3 This is a diagram of the similarity calculation operation in an embodiment of the present invention.

[0014] Figure 4 Schematic diagram of an embedded feature enhancer in an embodiment of the present invention.

[0015] Figure 5Schematic diagram of a hierarchical feature refiner in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] like Figure 1 As shown in FIG, a flow chart of a small sample target recognition method based on virtual feature enhancement and feature refinement of the present invention, the specific steps are as follows:

[0018] Step 1: In the training phase, a feature extractor is used to extract low-dimensional feature maps from limited SAR training samples;

[0019] like Figure 2 As shown in the overall framework of the method of the present invention, a feature extractor is implemented by a simple deep convolutional neural network (CNN) or transformer, and a set of SAR images is set.

[0020] Among them, C represents the total number of categories of the group of images, and B represents the number of samples in each category. Represents the jth sample in the i-th class. The low-dimensional feature map obtained after the sample passes through the feature extractor is represented as

[0021] Step 2: Using the embedded feature enhancer, we calculate the similarity and search for features far from the class center, obtain dissimilar feature pairs within the class and similar feature pairs between classes, and construct and enhance virtual features between features far from the class center.

[0022] The role of the embedded feature enhancer is to utilize the relative distribution information of features far away from the class center and enhance the characteristics of intra-class compactness and inter-class separation of features by enhancing virtual features.

[0023] In this process, if Figure 3 As shown in the calculation operation diagram of similarity, the concept of sample similarity is introduced, as shown in formula (1):

[0024]

[0025] in, Representation sample and The similarity between them, ||·||2 represents the L2 norm operation.

[0026] Set the class center sample of class i to be According to the sample similarity, the most dissimilar pairs within the class and the most similar pairs between classes are searched. The process is as follows: Figure 4 As shown in section S1.

[0027] Based on the obtained most dissimilar pairs within the class and the most similar pairs between the classes, a random weight parameter is used to construct a virtual feature. The label value of the virtual feature is also weighted by this random weight parameter. The process is as follows Figure 4 As shown in Section S2.

[0028] The construction of virtual features is shown in formula (2):

[0029]

[0030] in, Represented by similar pairs The constructed virtual features, α∈[0,1], represent weighting parameters and obey Gaussian distribution.

[0031] Step 3: A dynamic hierarchical feature refiner is used to first enhance the class discriminability of local features. Then, a convolution kernel is dynamically generated based on the unique features of the input SAR image. The discriminative local features of different dimensions are integrated into the global features, improving the intra-class compactness and inter-class separability of features.

[0032] The dynamic hierarchical feature refiner is used to further enhance the intra-class compactness and inter-class separability of features by integrating more discriminative features in the hierarchical structure and adaptively generating kernel functions based on the input SAR image. The structure of the dynamic hierarchical feature refiner is shown in Figure 2. Figure 5 As shown in Figure 3, it mainly consists of two parts: local feature enhancement and global feature enhancement.

[0033] Given a sample feature map First, perform local enhancement to obtain a local mask to enhance features from the spatial aspect. First, use a convolutional layer with batch normalization and nonlinear activation function to It is refined and then another convolutional layer with a Sigmoid activation function is used to generate a local mask.

[0034] Locally enhanced feature maps It is obtained by multiplying the local mask and the original feature map element by element. The calculation process is as follows:

[0035]

[0036] Among them, ⊙ represents the element-by-element product, represents a convolutional layer with batch normalization and nonlinear activation function with C channels, Represents a convolutional layer with a Sigmoid activation function.

[0037] Then the sample feature map Perform global enhancement and obtain a global mask to enhance features from the channel aspect. Figure 5 As shown in the middle right half, The global vector of is obtained by adaptive average pooling, and the adaptive convolution kernel is obtained by passing through a convolution layer with batch normalization and nonlinear activation function and a convolution layer with SoftMax. j ) is convolved with the generated convolution kernel to obtain the final global mask.

[0038] The generation process of the adaptive convolution kernel is as follows:

[0039]

[0040] Among them, k gen represents the generated adaptive convolution kernel, sf(·) represents the SoftMax function, f 1D (·) represents the one-dimensional convolution function, represents a one-dimensional convolution function with batch normalization and nonlinear activation function, and AAP(·) represents adaptive average pooling;

[0041] The calculation process of the dynamic hierarchical feature refiner is as follows:

[0042]

[0043] in, represents the final output of the dynamic hierarchical feature refiner, * represents the convolution function;

[0044] Through local and global enhancement, the dynamic hierarchical refiner integrates more discriminative features by mining the intrinsic structure of local features and global features in the data.

[0045] Because the scattering characteristics of SAR images are affected by multiple factors such as the imaging platform and azimuth, their dynamic changes are complex. The dynamic hierarchical refiner proposed in this embodiment can dynamically calculate the convolution kernel based on different inputs to handle the changing scattering characteristics of SAR images, thereby improving the accuracy and robustness of SAR ATR.

[0046] Step 4: Use a loss function that combines adaptive loss and recognition loss to optimize the network, improve recognition accuracy, and complete target recognition.

[0047] The loss function of this invention consists of two parts: recognition loss and adaptive loss. The adaptive loss pulls virtual features with different strengths based on the relative distribution of features, enhancing the features' intra-class compactness and inter-class separability. The recognition loss connects all virtual features with their corresponding similarity pairs and is calculated using a classifier.

[0048] Recognition loss L regAs shown in formula (6):

[0049]

[0050] Among them, y i Represents a C-dimensional vector, that is, the true sample label, Represents the predicted probability output by SoftMax, and B represents the number of samples in each class.

[0051] For the sample Set its similar inter-class features, dissimilar intra-class features, well-identified inter-class features, and well-identified intra-class features as

[0052] The base distance d of a well-identified pair bm Calculate by formula (7):

[0053]

[0054] Then, dynamic strength Calculated by formula (8):

[0055]

[0056] in, Respectively and The similarity between Representation sample The dynamic strength of ρ≥0 indicates that the bm Dynamically changing adaptive parameters.

[0057] Then the adaptive loss can be expressed by formula (9):

[0058]

[0059] Among them, L ada represents the adaptive loss.

[0060] For each feature pair, L ada It can be based on the distance of a difficult-to-identify pair and the reference distance d bm The difference between and provides dynamic strength. Therefore, the total loss function is expressed by Equation (10):

[0061] loss=λ1L reg +λ2L ada (10)

[0062] Among them, λ1 and λ2 represent weight coefficients, and the model parameters are updated by calculating the gradient of the loss function and back propagation.

[0063] like Figure 4 As shown in section S2, after this optimization step, the embedded feature enhancer leverages the relative distribution of features to supervise training and improve feature discriminability. These feature pair constraints theoretically act as regularizers in the hypothesis space, narrowing it and being key to improving SAR ATR performance with limited training samples.

[0064] In this embodiment, the recognition accuracy comparison data of the method of the present invention and other small sample target recognition methods are shown in Table 1.

[0065] Table 1

[0066]

[0067] Experimental results show that compared with other small sample target recognition methods, the method of the present invention can effectively improve the network recognition accuracy and robustness in small sample cases.

[0068] In summary, the method of the present invention mainly includes two parts: an embedded feature enhancer and a dynamic hierarchical refiner. The first part uses an embedded feature enhancer to search for features far away from the class, obtains intra-class dissimilar feature pairs and inter-class similar feature pairs, and uses the feature pairs to construct and enhance the corresponding virtual features, thereby increasing the amount of information used for supervised training. The second part uses a dynamic hierarchical feature refiner to integrate discriminative local features of different dimensions into global features by dynamically generating convolution kernels, thereby improving the intra-class compactness and inter-class separability of features. Finally, a loss function composed of adaptive loss and recognition loss is used to pull virtual features with different intensities, further improving the intra-class compactness and inter-class separability of features, reducing the hypothesis space, optimizing network parameters, and improving recognition accuracy. The method of the present invention effectively solves the problems of difficult data acquisition and high annotation costs, improves the learning effect of deep networks, and improves overall recognition accuracy. Compared with other small sample target recognition methods, the method of the present invention has higher recognition accuracy and robustness.

[0069] Those skilled in the art will appreciate that the embodiments described herein are intended to aid the reader in understanding the principles of the present invention, and it should be understood that the scope of the present invention is not limited to such specific descriptions and embodiments. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A small sample target recognition method based on virtual feature enhancement and feature refinement, the specific steps are as follows: Step 1: In the training phase, a feature extractor is used to extract low-dimensional feature maps from limited SAR training samples; Step 2: Using the embedded feature enhancer, we calculate the similarity and search for features far from the class center, obtain dissimilar feature pairs within the class and similar feature pairs between classes, and construct and enhance virtual features between features far from the class center. Step 3: A dynamic hierarchical feature refiner is used to first enhance the class discriminability of local features. Then, a convolution kernel is dynamically generated based on the unique features of the input SAR image. The discriminative local features of different dimensions are integrated into the global features, improving the intra-class compactness and inter-class separability of features. The step three is as follows: The dynamic hierarchical feature refiner mainly consists of two parts: feature local enhancement and feature global enhancement; Given a sample feature map First, local enhancement is performed to obtain a local mask to enhance features from the spatial aspect; first, a convolutional layer with batch normalization and nonlinear activation function is used to Perform refinement and then use another convolutional layer with Sigmoid activation function to generate a local mask; The local enhanced feature map is obtained by multiplying the local mask and the original feature map element by element The calculation process is as shown in formula (1): Among them, ⊙ represents the element-by-element product, represents a convolutional layer with batch normalization and nonlinear activation function with C channels, Represents a convolutional layer with a Sigmoid activation function; Then the sample feature map Perform global enhancement and obtain a global mask to enhance features from the channel aspect; obtain The global vector of is obtained by passing through a convolution layer with batch normalization and nonlinear activation function and a convolution layer with SoftMax to obtain an adaptive convolution kernel; the sample feature map Perform convolution operation with the generated convolution kernel to obtain the final global mask; The generation process of the adaptive convolution kernel is as follows: Among them, k gen represents the generated adaptive convolution kernel, sf(·) represents the SoftMax function, f 1D (·) represents the one-dimensional convolution function, represents a one-dimensional convolution function with batch normalization and nonlinear activation function, and AAP(·) represents adaptive average pooling; The calculation process of the dynamic hierarchical feature refiner is as follows: in, represents the final output of the dynamic hierarchical feature refiner, * represents the convolution function; Through local and global enhancement, the dynamic hierarchical refiner mines the intrinsic structure of local and global features in the data and integrates more discriminative features; Step 4: Use a loss function that combines adaptive loss and recognition loss to optimize the network, improve recognition accuracy, and complete target recognition.

2. The small sample target recognition method based on virtual feature enhancement and feature refinement according to claim 1, characterized in that: The step 1 is specifically as follows: Implement the feature extractor through a simple deep convolutional neural network CNN or transformer, set a set of SAR images Among them, C represents the total number of categories of the group of images, and B represents the number of samples in each category. Represents the jth sample in the i-th class. The low-dimensional feature map obtained after the sample passes through the feature extractor is represented as 3. The small sample target recognition method based on virtual feature enhancement and feature refinement according to claim 1, characterized in that: The step 2 is specifically as follows: The concept of sample similarity is introduced, as shown in formula (4): in, Representation sample and The similarity between them, ||·||2 represents the L2 norm operation; Set the class center sample of class i to be According to the sample similarity, the most dissimilar pairs within the class and the most similar pairs between classes are searched; Based on the obtained most dissimilar pairs within the class and the most similar pairs between classes, a random weight parameter is used to construct a virtual feature, and the label value of the virtual feature is generated by weighting with the random weight parameter; The construction of virtual features is shown in formula (5): in, Represented by similar pairs The constructed virtual features, α∈[0,1], represent weighting parameters and obey Gaussian distribution.

4. The small sample target recognition method based on virtual feature enhancement and feature refinement according to claim 1, characterized in that: The step 4 is specifically as follows: Recognition loss L reg As shown in formula (6): Among them, y i Represents a C-dimensional vector, that is, the true sample label, Represents the predicted probability output by SoftMax, and B represents the number of samples in each class; For the sample Set its similar inter-class features, dissimilar intra-class features, well-identified inter-class features, and well-identified intra-class features as The base distance d of a well-identified pair bm Calculate by formula (7): Then, dynamic strength Calculated by formula (8): in, Respectively and The similarity between Representation sample The dynamic strength of ρ≥0 indicates that the bm Dynamically changing adaptive parameters; Then the adaptive loss can be expressed by formula (9): Among them, L ada represents the adaptive loss; For each feature pair, L ada Based on a distance of a difficult pair and a reference distance d bm The difference between the two provides dynamic strength; The total loss function is expressed by formula (10): loss=λ1L reg +λ2L ada (10) Among them, λ1 and λ2 represent weight coefficients, and the model parameters are updated by calculating the gradient of the loss function and back propagation.

Images