A small sample fine-grained ship target recognition method based on feature reconstruction network

Abstract

The application belongs to the field of intelligent processing of space-based remote sensing ship targets, and particularly relates to a small sample fine-grained ship target recognition method based on feature reconstruction network, S1, taking large sample category ship target data as base class data for model training process; taking small sample category ship target data as new class data for model testing process; constructing support set and query set; S2, performing forward feature reconstruction on query set features through support set features; S3, performing reverse feature reconstruction on support set features through query set features; S4, calculating cycle consistency reconstruction loss, and training network model based on the loss; S5, calculating cycle consistency reconstruction error, and judging the category of the query set based on the error. The cycle consistency feature reconstruction network provided by the application has strong feature distance measurement capability, and can effectively improve the precision of small sample fine-grained ship target recognition.

Description

Technical Field

[0001] This invention belongs to the field of space-based remote sensing intelligent processing, specifically involving a small-sample fine-grained ship target recognition method based on feature reconstruction networks. Background Technology

[0002] Fine-grained target recognition aims to identify the specific type of target in remote sensing images, and it has extremely wide applications in both military and civilian fields. For example, the specific model of a naval vessel is crucial for assessing its threat level. However, images of some types of naval vessels are very scarce, and deep learning-based target recognition models suffer from severe overfitting to these types of vessels, resulting in extremely low recognition performance for small sample sizes. Therefore, fine-grained naval target recognition in remote sensing images with small sample sizes is a highly challenging problem.

[0003] To address the small sample size problem, meta-learning-based methods have been proposed in recent years. Meta-learning aims to learn general knowledge from a large amount of base class data and then transfer this knowledge to new class data with small sample sizes. Current meta-learning methods can be mainly divided into two categories: optimization-based and metric-based methods. Optimization-based methods first learn the optimal initial values ​​of the model based on the base class data and then fine-tune them based on the new class data. Metric-based methods aim to learn a general distance metric function from the base class data and directly apply this distance metric function to the new class data. Optimization-based methods have lower flexibility and overall accuracy because they require fine-tuning on the new class data; metric-based methods do not require fine-tuning the network on the new class data, resulting in higher overall accuracy and greater flexibility. The method proposed in this invention is based on metric learning.

[0004] Currently, metric learning-based methods can be mainly categorized into global feature-based, local feature-based, and attention-based methods. Global feature-based methods first pool both the support and query feature maps into global feature vectors, then use the distance between these global feature vectors as the distance between the support and query feature maps. Local feature-based methods treat the feature map as a set of spatial depth feature descriptors, aiming to design a suitable distance metric function to measure the distance between these sets. Attention-based methods aim to introduce an attention mechanism to allow the network to automatically focus on important feature parts. Among these, global feature-based methods lose a significant amount of detailed information in the feature map, greatly limiting the performance of fine-grained target recognition; while local feature-based methods largely preserve the detailed information in the feature map, making them highly suitable for small-sample, fine-grained ship target recognition tasks.

[0005] Among local feature-based methods, feature reconstruction networks are a high-performance approach. This method first reconstructs query features based on supporting features, then uses the reconstruction error between the reconstructed query features and the true query features as a distance metric between the supporting and query sets. However, current feature reconstruction networks only consider forward reconstruction (reconstruction of query features from supporting features), neglecting backward reconstruction (reconstruction of supporting features from query features). A single forward reconstruction error cannot effectively measure the similarity between supporting and query features. Intuitively, the more similar the supporting and query features are, the smaller both the forward and backward reconstruction errors should be. If the forward reconstruction error is small while the backward reconstruction error is large, the supporting and query features are not similar enough. Therefore, existing feature reconstruction networks suffer from a "metric bias" problem because they only use the forward reconstruction error as a distance metric. Furthermore, current feature reconstruction networks only consider the forward reconstruction loss during training, causing the model to overemphasize the forward reconstruction process and neglect the backward reconstruction process, lacking consistent reconstruction capabilities. This also results in a "reconstruction bias" problem, further affecting the accuracy of backward reconstruction. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of "reconstruction bias" and "metric bias" in traditional feature reconstruction networks, and to provide a small-sample ship target recognition method based on a cyclic consistency feature reconstruction network, which can be applied to the field of intelligent processing of space-based remote sensing ship targets.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a small-sample fine-grained ship target recognition method based on feature reconstruction networks, comprising the following steps:

[0008] S1. Use large-sample class ship target data as base class data for model training; use small-sample class ship target data as new class data for model testing; construct support set and query set;

[0009] S2. Reconstruct the query set features using the support set features;

[0010] S3. Reconstruct the support set features by using the query set features;

[0011] S4. Calculate the cycle consistency reconstruction loss and train the network model based on the loss;

[0012] S5. Calculate the cycle consistency reconstruction error and determine the category of the query set based on the error.

[0013] Specifically:

[0014] In step S1, the support set images and query set images are processed by a convolutional neural network to obtain the support set features and query set features.

[0015] In step S2, the support set features obtained in step S1 are used to reconstruct the query set features using ridge regression.

[0016] In step S3, the support set features are reconstructed based on the query set features obtained in step S1 using ridge regression.

[0017] In step S4, the reconstructed query set features and reconstructed support set features obtained in steps S2 and S3 are used to obtain the cycle consistency reconstruction loss, and the cycle consistency feature reconstruction network is optimized and trained based on this loss.

[0018] In step S5, the cyclic consistency reconstruction error is obtained based on the reconstructed query set features and reconstructed support set features obtained in steps S2 and S3, and the category of the query set image is determined based on this error.

[0019] This invention provides a small-sample fine-grained ship target recognition method based on feature reconstruction networks. The proposed cyclic consistency feature reconstruction network has a strong feature distance measurement capability, which can effectively improve the accuracy of small-sample fine-grained ship target recognition. Attached Figure Description

[0020] To more clearly illustrate the purpose, design concept, and innovation of the proposed method for reconstructing cyclic consistency features in networks, the invention will be described in detail below with reference to the accompanying drawings and tables.

[0021] Figure 1 This is a flowchart of the cyclic consistency feature reconstruction network method proposed in this invention.

[0022] Figure 2 This is a structural diagram of the reconstructed network based on the cyclic consistency feature proposed in this invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0024] Example 1

[0025] This embodiment is a small-sample fine-grained ship target recognition method based on feature reconstruction networks provided by the present invention. The method flow is as follows: Figure 1 As shown, it includes the following steps:

[0026] S1. Use large-sample class ship target data as base class data for subsequent model training; use small-sample class ship target data as new class data for subsequent model testing; construct support set and query set.

[0027] We construct the support set and query set using the standard N-way K-shot task setup in meta-learning. For an N-way K-shot task, a task set consists of N categories, each category includes K support set images, and 15 query set images are set for training and inference.

[0028] S2. Reconstruct the query set features using the support set features.

[0029] Suppose that the support set, after being processed by a convolutional neural network, yields support set features S. c ∈R kr×d Where k is the number of shots, r is the resolution of the feature map, and d is the number of channels in the feature map; after the query set is processed by the convolutional neural network, the resulting query set features are Q∈R. r×d The feature forward reconstruction process is as follows:

[0030] Q = W f S c (1)

[0031] Among them, W f Let be the forward reconstruction matrix. To efficiently solve for the forward reconstruction matrix, we need to find the following objective function:

[0032]

[0033] Where ||·|| is the Frobenius norm, λ f This is the regularization parameter.

[0034] Optimal forward reconstruction matrix The following can be calculated:

[0035]

[0036] in, S represents c The transpose of , where I represents the identity matrix.

[0037] Optimal Reconstruction Query Set Features The following can be calculated:

[0038]

[0039] Forward feature reconstruction transforms support set features into query set features using linear regression. The similarity between the support set features and the query set features can then be measured by comparing the reconstruction error between the reconstructed query set features and the true query set features. The reconstruction error between query set features reconstructed from support set features of the same category and the true query set features should be small, while the reconstruction error from support set features of different categories should be larger.

[0040] However, although forward feature reconstruction can measure the similarity between supporting features and query features to some extent, a network with excellent reconstruction capabilities should not only have forward reconstruction capabilities, but also reverse reconstruction capabilities.

[0041] S3. Reconstruct the support set features by using the query set features.

[0042] The feature reverse reconstruction process can be represented as follows:

[0043] S c =W b Q (5)

[0044] Among them, W b Let be the reverse reconstruction matrix. To efficiently solve for the reverse reconstruction matrix, we need to find the following objective function:

[0045]

[0046] Where ||·|| is the Frobenius norm, λ b This is the regularization parameter.

[0047]

[0048] in, Let Q represent the optimal reverse reconstruction matrix. T This represents the transpose of Q.

[0049] Reconstructing the support set features S c The following can be calculated:

[0050]

[0051] S4. Calculate the cycle consistency reconstruction loss and train the network model based on the loss.

[0052] To enhance the network's ability to perform cyclic consistency reconstruction, a cyclic consistency reconstruction loss function is proposed. Unlike traditional feature reconstruction networks that simply use the forward reconstruction loss as the classification loss, this invention uses the sum of the forward reconstruction loss and the backward reconstruction loss as the network's cyclic consistency reconstruction loss. First, the target classification probability based on the forward reconstruction error is defined as follows:

[0053]

[0054] Where, x q y represents the query set of images. q The tags represent the images in the query set, and γ represents the temperature coefficient. The query feature reconstruction error is represented by the following definition:

[0055]

[0056] The target classification probability based on the inverse reconstruction error is defined as follows:

[0057]

[0058] The supporting feature reconstruction error is defined as follows:

[0059]

[0060] The final calculation of the cycle-consistent reconstruction loss is as follows:

[0061] L = CE(y q ,p f )+β·CE(y q ,p b (13)

[0062] Where β represents the loss fusion coefficient, which is set to 0.1 in this invention; CE represents the cross-entropy loss function, which is defined as follows:

[0063] CE(y,p)=-∑y i log(p i (14)

[0064] S5. Calculate the cycle consistency reconstruction error and determine the category of the query set based on the error.

[0065] Cyclic Consistency Reconstruction Error d c The calculation formula is as follows:

[0066]

[0067] By reconstructing the error based on cycle consistency, the category j of the images in the query set can be determined.

[0068]

[0069] Where C represents the total number of categories.

[0070] Example 2

[0071] Building upon Example 1, this embodiment selects the publicly available fine-grained ship target dataset FGSCR to experiment with the proposed Cyclic Consistency Reconstruction Network, verifying its feasibility and effectiveness. The FGSCR dataset contains 9320 images across 42 ship categories. The image sizes in the FGSCR dataset range from 50×50 to 1500×1500 pixels, with different aspect ratios for the ships. Furthermore, the aspect ratios of ships within the same category may also vary significantly, making the FGSCR dataset highly suitable for fine-grained ship classification tasks with high intra-class variability and low inter-class variability. This invention selects 25 ship image categories with a relatively large number for experiments, with 15 categories used as the training set, 5 as the validation set, and 5 as the test set. In addition, this embodiment selects nine novel few-shot target recognition algorithms as comparison methods.

[0072] Table 1 presents the quantitative experimental results of the classification algorithm proposed in this invention and the above 9 advanced methods on this dataset.

[0073] Table 1. Accuracy (%) of different target recognition algorithms for small sample datasets in the FGSCR dataset

[0074]

[0075]

[0076] According to Table 1, the present invention achieved an accuracy of 52.01% in 1-shot scenarios and 79.13% in 5-shot scenarios. Compared with the baseline method FRN, the present invention achieved a performance improvement of 6.54% in 1-shot scenarios and 7.93% in 5-shot scenarios. Compared with the state-of-the-art fine-grained classification method for small-sample remote sensing images, CWFRN, the present invention achieved a performance improvement of 2.04% in 1-shot scenarios and 4.67% in 5-shot scenarios.

[0077] This invention addresses the field of intelligent processing of space-based remote sensing ship targets by proposing a small-sample ship target recognition method based on a cyclic consistency feature reconstruction network. Experimental results demonstrate the effectiveness of the algorithm in small-sample, fine-grained ship target recognition.

Claims

1. A small-sample, fine-grained ship target recognition method based on feature reconstruction networks, characterized in that, Includes the following steps: S1. Ship target data of large sample categories are used as base class data for model training; ship target data of small sample categories are used as new class data for model testing. Construct the support set and query set, and obtain the support set features and query set features; S2. Reconstruct the query set features using the support set features; S3. Reconstruct the support set features by using the query set features; The feature reverse reconstruction process is represented as follows: ， Where Q represents the query set feature. To reconstruct the matrix in reverse, solve for the following objective function: ， Among them, S c为 Support set features It is the Frobenius norm. For regularization parameters; ， in, Represents the optimal reverse reconstruction matrix. express Transpose of; Reconstructing support set features The calculation is as follows: ； S4. Calculate the cycle consistency reconstruction loss and train the network model based on the loss; The sum of the forward reconstruction loss and the backward reconstruction loss is used as the recurrent consistency reconstruction loss of the network. First, the target classification probability based on the forward reconstruction error is defined as follows: ， in, This indicates a query set of images. Tags representing the images in the query set. Indicates the temperature coefficient. The query feature reconstruction error is represented by the following definition: ； The target classification probability based on the inverse reconstruction error is defined as follows: ， The supporting feature reconstruction error is defined as follows: ， The final calculation of the cycle-consistent reconstruction loss is as follows: ， in, Indicates the loss fusion coefficient; The cross-entropy loss function is defined as follows: ； S5. Calculate the cycle consistency reconstruction error and determine the category of the query set based on the error; Cyclic Consistency Reconstruction Error The calculation formula is as follows: ； By reconstructing the error based on cycle consistency, the category of images in the query set can be determined. : ， in, This represents the total number of categories.

2. The method for small-sample fine-grained ship target recognition based on feature reconstruction networks according to claim 1, characterized in that, In step S1, the support set images and query set images are processed by a convolutional neural network to obtain the support set features and query set features.

3. A small-sample fine-grained ship target recognition method based on feature reconstruction networks according to claim 1 or 2, characterized in that, In step S2, the support set features obtained in step S1 are used to reconstruct the query set features using ridge regression.

4. The method for small-sample fine-grained ship target recognition based on feature reconstruction networks according to claim 3, characterized in that, In step S3, the support set features are reconstructed based on the query set features obtained in step S1 using ridge regression.

5. The method for small-sample fine-grained ship target recognition based on feature reconstruction networks according to claim 4, characterized in that, In step S4, the reconstructed query set features and reconstructed support set features obtained in steps S2 and S3 are used to obtain the cycle consistency reconstruction loss, and the cycle consistency feature reconstruction network is optimized and trained based on this loss.

6. The method for small-sample fine-grained ship target recognition based on feature reconstruction networks according to claim 5, characterized in that, In step S5, the cyclic consistency reconstruction error is obtained based on the reconstructed query set features and reconstructed support set features obtained in steps S2 and S3, and the category of the query set image is determined based on this error.

Images