Small sample target recognition method and network based on spatial representation learning and reinforcement

Abstract

The application belongs to the technical field of image processing, and discloses a small sample target recognition method and network based on space representation learning and reinforcement, wherein the network model is trained through steps of feature extraction, contrast pair construction, feature mutual enhancement, space granularity comparison, comparison loss calculation, attention pooling, query set sample recognition and loss optimization, and accurate recognition results can be obtained for input images to be recognized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to deep learning and small sample image recognition technology, in particular to a small sample target recognition method and network based on spatial representation learning and reinforcement. BACKGROUND

[0002] In the big data era, the ability of humans to collect, store, transmit and manage data is increasing, and a large amount of data resources are accumulated. How to quickly and effectively analyze massive data has become a difficult problem. Under this background, deep learning has attracted widespread attention from academia and industry due to its powerful feature extraction capability, and has achieved great success in the field of computer vision. Among them, target recognition, as the most basic and important task in computer vision, is widely studied. However, the deep learning technology relying on big data still faces great challenges. Traditional deep learning needs a large number of labeled samples for training. When the samples are insufficient, the performance of the model will be severely degraded, causing overfitting problem. In reality, it takes a lot of manpower and material resources to establish a standard data set, or it is difficult to collect a large number of labeled samples. For example, in the field of medical image processing, it is very difficult to collect samples of rare diseases; in the field of marine target recognition, the complexity of the marine system makes it difficult to obtain a large number of samples of some marine targets. Therefore, more and more small sample target recognition methods have been proposed.

[0003] The current mainstream small sample target recognition methods mainly fall into two categories: data augmentation-based methods and knowledge transfer-based methods. The data augmentation-based method, as the name implies, is to expand or enhance the features of the original small sample data set with the help of auxiliary data or auxiliary information in the case of limited data. The knowledge transfer-based method is to transfer the knowledge learned on the data-rich base data set to the data-scarce new class data set. According to the knowledge to be transferred, it can be further divided into small sample target recognition methods based on measurement and small sample target recognition methods based on optimization. Among them, the measurement-based method has the advantages of simplicity, efficiency and good recognition effect, and is the most effective method to solve the problem of small sample target recognition. The main idea of this method is to train the model by simulating the small sample scene on the base data set, so that the model learns how to extract effective features based on a small number of samples, and identifies the query set samples by measuring the similarity between the support set prototype and the query set sample features. Then, the knowledge learned on the base class is transferred to the new class for testing. Therefore, how to extract rich and effective features based on a small number of samples, and how to construct accurate class prototypes based on the extracted features are the two key problems of this method.

[0004] However, the existing small sample recognition method based on metrics often only represents the image based on the global features of the support set samples and the query set samples when training the base class model, and similarity measurement is performed based on this to realize label prediction of the query set samples, and then the cross entropy loss is calculated by using the predicted value of the query set samples and the real value thereof to update the model parameters in reverse propagation. More potential feature information in the image is not considered, resulting in that the feature representation extracted by the model is not rich enough, and it is difficult to effectively represent the class. SUMMARY

[0005] In view of the problems in the prior art, the small sample target recognition method and network based on spatial representation learning and reinforcement are provided, and improvements are made from the following aspects: in view of the problem that the number of samples is small in the feature extraction process of the small sample target recognition task, it is difficult to extract rich and effective feature representation, the feature representation learning and reinforcement method based on spatial granularity comparison and attention pooling is proposed, the class feature information contained in each spatial granularity position of the sample image is fully utilized and mined, and the spatial granularity region with more information is highlighted and reinforced; in the process of calculating the spatial comparison loss, a feature enhancement mechanism is designed to highlight the region with greater influence on similarity in the feature map, so that the similarity calculation of the corresponding spatial positions of the two feature maps is more accurate; at the same time, the attention pooling is used instead of the global average pooling in the pooling operation, the spatial granularity region with better discriminability is highlighted, the spatial information expression ability is further strengthened, and more rich feature information can be captured.

[0006] To solve the above technical problems, the technical scheme adopted by the present application is:

[0007] A small sample target recognition method based on spatial representation learning and reinforcement, comprising the following steps:

[0008] Step 1, constructing a small sample recognition task:

[0009] In each iteration training, first, a small sample recognition task is randomly sampled from the base class dataset to simulate a small sample recognition scene, specifically, N classes are randomly selected from the base class, and K samples are sampled from each class to form a support set S; and a plurality of samples are sampled from each class to form a query set Q;

[0010] Step 2, feature extraction:

[0011] A convolutional neural network is used as a feature extraction network to extract features of the input sample image, specifically, all support set samples and query set samples are input into the feature extraction network f θ with parameters θ to obtain corresponding feature representations f θ (x), to obtain support set feature maps and query set feature maps;

[0012] Step 3, constructing contrast pairs:

[0013] Based on the support set feature map and the query set feature map obtained in step 2, construct contrast pairs. Specifically, for all support set samples and query set samples, which have corresponding true class labels, two samples of the same class are regarded as a pair of positive sample pairs, and two samples of different classes are regarded as a pair of negative sample pairs. Construct sample pairs, and then calculate the spatial granularity similarity of the two sample feature maps in each sample pair.

[0014] Step 4: Feature mutual enhancement:

[0015] For two samples i and j in a sample pair, both have corresponding feature maps f θ (i) and f θ (j) obtained in step 2. Use f θ (i) to enhance f θ (j) to obtain the enhanced feature map Specifically: f θ (i) is subjected to global average pooling operation and multiplied with the feature map f θ (j) channel by channel to obtain the weighted feature map The feature map is subjected to spatial attention processing to obtain the attention map The attention map is multiplied with the feature map f θ (j) of sample j element by element to obtain the enhanced feature map

[0016] Use f θ (j) to enhance f θ (i), and use f θ (j) to enhance f θ (i). The operation steps are the same, and the enhanced feature map is obtained

[0017] Step 5: Spatial granularity comparison, calculate contrast loss:

[0018] Based on the original feature map and the enhanced feature map, perform spatial granularity comparison. Specifically, take the spatial granularity comparison between the original feature map f θ (i) of sample i and the feature map of sample j based on the enhanced feature map of sample i as an example. Calculate the similarity between each spatial position and the other feature map on the two feature maps, and sum and average the similarity between all spatial positions and the feature map to obtain the spatial granularity similarity between the two feature maps, denoted as

[0019] Similarly, calculate the spatial granularity similarity between the original feature map f θ (i) of sample j and the feature map of sample i based on the enhanced feature map of sample j.θ (j) the feature map of sample i based on the enhanced feature map of sample j the spatial granularity similarity between sample i and sample j is denoted as sim(f

[0020] The average of the two similarities is taken as the final spatial granularity similarity between sample i and sample j, denoted as sim(f θ (i),f θ (j));

[0021] For sample i, a spatial granularity contrast loss function is constructed according to the spatial granularity similarities of its positive and negative sample pairs, denoted as L The spatial granularity contrast losses of all samples are added together as the total spatial contrast loss, denoted as L SC ;

[0022] Step 6: Attention Pooling:

[0023] Attention pooling operations are used on all support set and query set sample feature maps extracted in step 2 to convert them into support set feature vectors and query set feature vectors;

[0024] Step 7: Query Set Sample Recognition:

[0025] Each class prototype is constructed using the corresponding support set sample feature vector of each class, then the similarity between each query set sample and each class prototype is calculated, and the Softmax function is used to calculate the probability value of the query set sample belonging to each class, and the class with the maximum probability value is taken as the predicted label of the query set sample. Then, the cross-entropy loss is calculated according to the true label of the query set sample as the small sample classification loss, denoted as L FSC ;

[0026] Step 8: Loss Optimization:

[0027] The spatial contrast loss L SC and the small sample classification loss L FSC are combined to form the total loss of the entire training process, denoted as L, i.e.

[0028] L = α SC L SC + α FSC L FSC

[0029] where α SC and α FSC are hyperparameters used to adjust the weights of L SC and L FSC .

[0030] Furthermore, in step 1, specifically, N categories are randomly selected from the base classes, and K samples are collected from each category to form a support set S. Each category is further sampled to form a query set Q. in, This represents the i-th support set sample in the support set S. Indicates support set samples The corresponding true labels, N×K represent the number of support set samples in the support set S; This represents the i-th query set sample in query set Q. Represents a query set sample The corresponding real label, M represents the number of query set samples in query set Q.

[0031] Furthermore, in step 3, for a certain support set sample... For example, it has K-1 support set samples and several query set samples of the same category. It forms several positive sample pairs with these samples and forms negative sample pairs with the remaining samples of different categories; the same applies to the remaining samples.

[0032] In the few-sample recognition task constructed in each iteration of training, for any sample i, N×K+M-1 positive and negative sample pairs can be constructed for it.

[0033] Furthermore, in step 4, f is used θ (i) to enhance f θ Taking (j) as an example, firstly, for feature map f θ (i) Obtain a weight vector v using global average pooling. i , will v i With feature map f θ (j) Multiplying each channel sequentially yields the weighted feature map. Right now This feature map It contains feature information about the category to which sample i belongs; next, the feature map... The input is sent to a spatial attention module A, and the output is a single-channel attention map of size H×W×1. Right now Where, θ A The parameters in spatial attention module A are represented by H (height), W (width), and C (number of channels); the output attention map is... Feature map f of sample j θ (j) Perform element-wise multiplication to obtain the enhanced feature map. Right now Where ⊙ represents element-wise multiplication.

[0034] Furthermore, in step 5, a spatial granularity comparison is performed based on the original feature map and the enhanced feature map. Specifically, the original feature map f of sample i is compared. θ (i) and the enhanced feature map of sample j based on sample i Taking the spatial granularity comparison as an example, both feature maps are of size H×W×C. For each feature map, there are H×W pixels at all spatial locations. Each spatial pixel p has a corresponding C-dimensional feature vector along the channel direction, which are denoted as follows: and in, f represents the original feature map of sample i. θ (i) The feature vector in the channel direction corresponding to the p-th spatial pixel (p∈[1,HW]); This represents the feature map of sample j based on sample i. The feature vector corresponding to the p-th (p∈[1,HW]) spatial pixel in the channel direction;

[0035] Calculate the similarity between each spatial location on each of the two feature maps and the other feature map, and then calculate the similarity between feature map f and the other feature map. θ (i) Spatial location p and feature map When calculating the similarity, first analyze the feature map f. θ (i) Eigenvector at spatial location p and feature maps Feature vectors of all spatial locations After normalization, the feature maps f are calculated using vector dot products. θ (i) Eigenvector at spatial location p and feature map Feature vectors at various spatial locations The similarity between them is used, and the highest similarity value is taken as the feature map f. θ (i) Spatial location p and feature map The similarity, that is,

[0036]

[0037] The similarity calculation between other spatial locations and feature maps is similar;

[0038] Finally, the spatial granularity similarity between the feature maps is obtained by summing and averaging the similarities between all spatial locations and the feature maps, denoted as . Right now,

[0039]

[0040] Similarly, the original feature map f of sample j is calculated. θ(j) the feature map of sample i based on the enhanced feature map of sample j the spatial granularity similarity between sample i and sample j, denoted as sim(f i.e.,

[0041]

[0042] wherein, fj represents the original feature map of sample j θ (j) the feature vector in the channel direction corresponding to the pth (p e [1, HW]) spatial pixel point of sample i; (j) the feature map of sample i based on the enhanced feature map of sample j (j) the feature vector in the channel direction corresponding to the pth (p e [1, HW]) spatial pixel point of sample i;

[0043] The average of the two spatial granularity similarities obtained is taken as the final spatial granularity similarity between sample i and sample j, denoted as sim(f θ (i), f θ (j)), i.e.,

[0044]

[0045] The spatial granularity similarities of the remaining sample pairs are calculated in steps 4 and 5.

[0046] Further, in step 5, for sample i, a spatial granularity contrast loss function is constructed according to the spatial granularity similarities of its positive and negative sample pairs, denoted as L i.e.,

[0047]

[0048] wherein, N(i) represents the number of samples of the same class as sample i; 1 cond is an indicator function that takes the value 1 when cond is true and 0 when it is not true; τ is a scalar temperature parameter; finally, the spatial granularity contrast losses of all samples are added together as the total spatial contrast loss, denoted as L SC i.e.,

[0049]

[0050] Further, a spatial contrast loss is obtained through step 5, and then, in step 6, a small sample classification loss is calculated by using query set samples, before the calculation of the classification loss, a spatial attention mechanism combined with pooling is used to convert the feature map into a feature vector, specifically, all support set and query set sample feature maps extracted in step 2 are converted into feature vectors by using attention pooling operation, taking any sample i as an example, for the sample i, the size of the feature map is HxWxC, first, the global average pooling is performed to obtain the global feature vector z and copy HxW parts, then the original feature map is merged in the channel direction to obtain the merged feature map, denoted as f θ (i), f θ (i) is input into the attention pooling module APool, and a series of convolutional layers are output with the original feature map f θ (i) spatial scale same attention map A i , that is, wherein, represents a convolutional layer in the attention pooling module, and φ represents a convolutional layer parameter; σ is a logic function; then, the attention map A i is used to weight and sum each spatial position of the original feature map f θ (i) to obtain the final feature vector, denoted as f i , The calculation formula is,

[0051]

[0052] The feature map of the remaining samples is converted into a feature vector in the same way.

[0053] The application also provides a small sample target recognition network based on spatial representation learning and reinforcement, which is used to implement the small sample target recognition method based on spatial representation learning and reinforcement as described above, and comprises a feature extraction network, a spatial representation learning module, a spatial representation reinforcement module and a loss calculation module; the feature extraction network is used to extract image features of input samples to obtain feature maps; the spatial representation learning module is used to construct positive and negative sample pairs for the feature maps extracted by the feature extraction network, and calculate a spatial contrast loss through a feature enhancement mechanism and spatial granularity contrast; the spatial representation reinforcement module is used to convert the feature maps extracted by the feature extraction network into feature vectors through attention pooling operation, perform similarity measurement, and output query set sample prediction labels; and the loss calculation module is used to calculate a small sample classification loss by using query set samples, and jointly calculate a total loss with the spatial contrast loss.

[0054] Compared with the prior art, the application has the following advantages:

[0055] In view of the problem that in the feature extraction process of a small sample target recognition task, the sample quantity is small and it is difficult to extract rich and effective feature representations, the application provides a feature representation learning and strengthening method based on the combination of spatial granularity comparison and attention pooling, fully utilizes and mines the category feature information contained in each spatial granularity position of a sample image, and highlights the spatial granularity region with a larger information quantity; in the process of calculating the spatial comparison loss, a feature enhancement mechanism is designed to highlight the region with a greater influence on the similarity in the feature map, so that the similarity calculation of the corresponding spatial positions of the two feature maps is more accurate; meanwhile, in the pooling operation, attention pooling is used instead of global average pooling to highlight the spatial granularity region with a greater discriminative ability, further strengthens the spatial information expression capability, can capture richer feature information, enriches the feature expression, and improves the target recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0057] Figure 1 The network structure diagram of the application. DETAILED DESCRIPTION

[0058] The application will be further described below in combination with the drawings and specific embodiments.

[0059] In combination with Figure 1 The embodiment provides a small sample target recognition method based on spatial representation learning and strengthening, which comprises the following steps:

[0060] Step 1, constructing a small sample recognition task:

[0061] In each iteration training, first, a small sample recognition task is randomly sampled from a base class data set (the base class data set is a category data with a relatively sufficient data quantity, such as the international public data set miniImageNet and tieredImageNet), a small sample recognition scene is simulated, and specifically, N categories are randomly selected from the base class, K samples are sampled from each category to form a support set S, a plurality of samples are sampled from each category to form a query set Q, Wherein, represents the i-th support set sample in the support set S, represents the support set sample corresponding to the true label, N x K represents the support set sample quantity in the support set S; denotes the i-th query set sample in the query set Q, denotes the query set sample corresponding ground truth label, M denotes the number of query set samples in the query set Q.

[0062] Step 2, feature extraction:

[0063] A convolutional neural network (such as ResNet) is used as a feature extraction network to extract features of the input sample images. Specifically, all support set samples and query set samples are input into the feature extraction network f θ with parameters θ to obtain corresponding feature representations f θ (x), obtaining support set feature maps and query set feature maps.

[0064] For example, for the i-th support set sample , after passing through the feature extraction network, the corresponding feature map is obtained. It is a tensor with size HxWxC, where H, W, and C represent the height, width, and number of channels of the feature map, respectively. The same applies to the query set samples.

[0065] Step 3, constructing contrast pairs:

[0066] Based on the support set feature maps and query set feature maps obtained in step 2, contrast pairs are constructed. Specifically, for all support set samples and query set samples, there are corresponding ground truth class labels. Two samples with the same class are considered as a pair of positive sample pairs, and two samples with different classes are considered as a pair of negative sample pairs. For example, for a certain support set sample S xi , there are K-1 support set samples and several query set samples with the same class as it. It forms several positive sample pairs with these samples, respectively, and forms negative sample pairs with the remaining samples with different classes. The same applies to the remaining samples.

[0067] Therefore, in each iteration of training the small sample recognition task constructed, for any sample i, NxK+M-1 positive and negative sample pairs can be constructed for it.

[0068] Next, the spatial granularity similarity of the feature maps of the two samples in each sample pair will be calculated.

[0069] Step 4: Feature mutual enhancement:

[0070] Before calculating the spatial granularity similarity, a feature mutual enhancement mechanism is designed to make the similarity calculation more accurate. Specifically, for two samples i and j in a sample pair, they both have corresponding feature maps f θ (i) and f θ (j) obtained in step 2, respectively. f θ(i) to enhance f θ (j) and use f θ (j) to enhance f θ (i) to obtain an enhanced feature map and an enhanced feature map

[0071] to enhance f θ (i) to enhance f θ (j) as an example, specifically: first, using a global average pooling operation on the feature map f θ (i) to obtain a weight vector v i , and multiplying v i with the feature map f θ (j) channel by channel to obtain a weighted feature map that is The feature map contains feature information of the class to which the sample i belongs; next, the feature map is input into a spatial attention module A, and an output is an attention map of a single channel with a size of HxWx1 that is where θ A represents parameters in the spatial attention module A, H is the height, W is the width, and C is the number of channels. The role of this module is to highlight important regions in the feature map f θ (j) related to the class feature of the sample i, so the value of the region in the attention map is greater when the region is more relevant to the class feature of the sample i; the output attention map is multiplied element by element with the feature map f θ (j) of the sample j to obtain an enhanced feature map that is where denotes an element-by-element multiplication operation.

[0072] to enhance f θ (j) to enhance f θ (i), and f θ (j) to enhance f θ (i) are the same, and an enhanced feature map

[0073] Step 5: Spatial granularity comparison, calculate the comparison loss:

[0074] After step 4, the enhanced feature map and are obtained. Based on the original feature map and the enhanced feature map, spatial granularity comparison is performed. Specifically, the original feature map f θ (i) of the sample i is compared with the enhanced feature map Taking spatial granularity comparison as an example, the similarity between each spatial location on each of the two feature maps and the other feature map is calculated. The summation and averaging of the similarities between all spatial locations and the feature maps yields the spatial granularity similarity between the two feature maps, denoted as .

[0075] Since both feature maps are of size H×W×C, they have H×W pixels at all spatial locations. Each spatial pixel p has a C-dimensional feature vector corresponding to the channel direction, which are denoted as follows: and in, f represents the original feature map of sample i. θ (i) The feature vector in the channel direction corresponding to the p-th spatial pixel (p∈[1,HW]); This represents the feature map of sample j based on sample i. The feature vector corresponding to the p-th spatial pixel (p∈[1,HW]) in the channel direction. Since the foreground object in the image can be located anywhere in the image, directly calculating the similarity based on the one-to-one correspondence of spatial locations may introduce errors. Therefore, a new spatial granularity similarity calculation method is designed, which calculates the similarity between each spatial location on the two feature maps and the other feature map.

[0076] For example, in calculating the feature map f θ (i) Spatial location p and feature map When calculating the similarity, first analyze the feature map f. θ (i) Eigenvector at spatial location p and feature maps Feature vectors of all spatial locations After normalization, the feature maps f are calculated using vector dot products. θ (i) Eigenvector at spatial location p and feature map Feature vectors at various spatial locations The similarity between them. The maximum similarity value is used as the feature map f. θ (i) Spatial location p and feature map The similarity, that is,

[0077]

[0078] The similarity calculation between other spatial locations and feature maps is performed similarly. Finally, the spatial granularity similarity between the feature maps is obtained by summing and averaging the similarities between all spatial locations and feature maps, denoted as . Right now,

[0079]

[0080] Similarly, the original feature map f θ (j) of sample j can be calculated. The spatial granularity similarity between the original feature map f i.e.

[0081]

[0082] wherein, f θ (j) of sample j. The feature vector corresponding to the pth (p ∈ [1, HW]) spatial pixel point in the channel direction of the enhanced feature map f i based on sample j.

[0083] The average of the two similarities obtained in the above steps is taken as the final spatial granularity similarity between sample i and sample j, denoted as sim(f θ (i), f θ (j)), i.e.

[0084]

[0085] The spatial granularity similarities of the remaining sample pairs are calculated according to the above steps 4 and 5.

[0086] Therefore, for sample i, a spatial granularity contrast loss function can be constructed according to the spatial granularity similarities of its positive and negative sample pairs, denoted as L i.e.

[0087]

[0088] wherein, N(i) represents the number of samples of the same class as sample i; 1 cond is an indicator function that takes the value 1 when cond is true and 0 when it is not; τ is a scalar temperature parameter.

[0089] Finally, the spatial granularity contrast losses of all samples are added together as the total spatial contrast loss, denoted as L SC i.e.

[0090]

[0091] The spatial contrast loss is used to narrow the distance between positive samples and widen the distance between negative samples at the spatial granularity level, so as to constrain the model to learn more spatial granularity features, mine potential details in the image, and further improve the feature extraction capability of the model.

[0092] Step 6: attention pooling:

[0093] On the basis of spatial representation learning, the attention pooling is combined to further strengthen the representation ability of the model to spatial information. After step 5, the spatial contrast loss is obtained. Next, the query set samples are used to calculate the small sample classification loss. Before calculating the classification loss, the feature map needs to be converted into a feature vector. The traditional small sample identification method based on measurement simply uses global average pooling to convert the feature map into a feature vector, which gives the same weight to the features at different spatial positions and ignores the difference in the contribution of features at different spatial positions. Therefore, the spatial attention mechanism is combined with the pooling to assign different attention parameters to the features at the spatial level during the pooling operation, and the attention parameters are used as the weight coefficients of the features at different spatial positions, so as to distinguish the contribution of the features at different spatial positions and further strengthen the representation ability of the model to spatial information.

[0094] Specifically, the attention pooling operation is used on all the support set and query set sample feature maps extracted in step 2 to convert them into support set feature vectors and query set feature vectors. Taking any sample i as an example, for the sample i, the size of the feature map is HxWxC. First, the global average pooling is performed to obtain a global feature vector z and copy HxW copies, and then the global feature vector z is merged with the original feature map in the channel direction to obtain a merged feature map, denoted as f θ (i), f θ (i) is input into the attention pooling module APool, and a series of convolution layers are used to output an attention map A θ (i) with the same spatial scale as the original feature map f i , that is, wherein, denotes a convolution layer in the attention pooling module, and φ denotes the parameters of the convolution layer; σ is a logistic function. Then, the attention map A i is used to weight and sum each spatial position of the original feature map f θ (i) to obtain a final feature vector, denoted as f i , The calculation formula is,

[0095]

[0096] The feature map of the remaining samples is converted into a feature vector in the same way.

[0097] Step 7: Query set sample identification:

[0098] After step 6, the feature vectors of all support set and query set samples in the small sample identification task can be obtained. Next, the prototype of each category is constructed using the feature vectors of the support set samples corresponding to each category, and then the similarity between each query set sample and each category prototype is calculated, and the probability value of the query set sample belonging to each category is calculated using the Softmax function, and the category with the maximum probability value is taken as the predicted label of the query set sample. Then, the cross-entropy loss is calculated according to the true label of the query set sample as the small sample classification loss, denoted as L FSC .

[0099] Step 8: Loss optimization:

[0100] The spatial contrast loss L SC is combined with the small sample classification loss L FSC to form the total loss of the entire training process, denoted as L, that is

[0101] L = a SC L SC + a FSC L FSC

[0102] Where a SC and a FSC are hyperparameters used to adjust the weights of L SC and L FSC .

[0103] Finally, the model is optimized using the gradient descent algorithm according to the total loss.

[0104] Iterate steps 1-8.

[0105] As another embodiment, a small sample target identification network based on spatial feature learning and reinforcement is also provided for implementing the small sample target identification method based on spatial feature learning and reinforcement as described above, including a feature extraction network, a spatial feature learning module, a spatial feature reinforcement module, and a loss calculation module. The feature extraction network is used to extract image features of input samples to obtain a feature map. The spatial feature learning module is used to construct positive and negative sample pairs for the feature map extracted by the feature extraction network, and calculate the spatial contrast loss through the feature enhancement mechanism and the spatial granularity contrast. The spatial feature reinforcement module is used to convert the feature map extracted by the feature extraction network into a feature vector through attention pooling operation, perform similarity measurement, and output the predicted label of the query set sample. The loss calculation module is used to calculate the small sample classification loss using the query set sample, and jointly calculate the total loss with the spatial contrast loss.

[0106] The image to be identified is input into the trained network model, and an identification result with high accuracy can be output.

[0107] In summary, for the key problem of small sample target recognition based on measurement, the application proposes a small sample target recognition method based on spatial feature learning and reinforcement. The strength of feature representation determines the performance of the target recognition model, and it is even more so for small sample target recognition. Most existing small sample recognition methods based on measurement only use global features to represent images when extracting features, resulting in insufficient feature expression. To solve this problem, the application proposes a feature representation learning and reinforcement method based on spatial granularity comparison and attention pooling, which fully utilizes and mines the class feature information contained in each spatial granularity position of the sample image, and highlights the spatial granularity area with more information. Therefore, the model can capture more rich feature information, improve the feature representation ability of the model, and improve the target recognition accuracy.

[0108] Of course, the above description is not a limitation on the application, and the application is not limited to the above examples. Within the scope of the application, changes, modifications, additions or substitutions made by those skilled in the art should be within the scope of the application.

Claims

1. A small sample target recognition method based on spatial representation learning and reinforcement, characterized in that, Comprising the following steps: Step 1, constructing a small sample recognition task: In each iteration training, first, a small sample recognition task is randomly sampled from the base class dataset to simulate a small sample recognition scene, specifically, N classes are randomly selected from the base class, and K samples are sampled from each class to form the support set S; and a plurality of samples are sampled from each class to form the query set Q; Step 2, feature extraction: The convolutional neural network is used as a feature extraction network to extract features of an input sample image, specifically, all support set samples and query set samples are input into the feature extraction network with parameters θ to obtain corresponding feature representations to obtain a support set feature map and a query set feature map;​ Step 3, constructing a contrast: Based on the support set feature map and the query set feature map obtained in step 2, a contrast is constructed, specifically, for all support set samples and query set samples, there are corresponding true class labels, two samples of the same class are regarded as a pair of positive sample pairs, and two samples of different classes are regarded as a pair of negative sample pairs, a sample pair is constructed, and then the spatial granularity similarity of the two sample feature maps in each sample pair is calculated; Step 4: feature mutual enhancement: For two samples i and j in a sample pair, both have corresponding feature maps obtained after step 2 and , are enhanced by to obtain enhanced feature maps , specifically: After global average pooling operation, multiply the feature map channel by channel to obtain the weighted feature map , the feature map is processed by spatial attention to obtain the attention map , the attention map is multiplied element by element with the feature map of sample j to obtain the enhanced feature map ;​ With to enhance , with to enhance the operation steps, get enhanced feature map ; Step 5: spatial granularity comparison, calculate contrast loss: Based on the original feature map and the enhanced feature map, spatial granularity comparison is performed, specifically, the original feature map of sample i is compared with the spatial granularity of the enhanced feature map of sample j based on sample i , and the similarity of each spatial position on the two feature maps with the other feature map is calculated respectively. The similarity between all spatial positions and the feature map is summed and averaged to obtain the spatial granularity similarity between the two feature maps, denoted as ; Similarly, the spatial granularity similarity between the original feature map of sample j and the enhanced feature map of sample i based on sample j is calculated, denoted as ;​​ The mean of the two similarities is taken as the final spatial granularity similarity between sample i and sample j, denoted as ; For sample i, the spatial granularity contrast loss function is constructed according to the spatial granularity similarity of its positive and negative sample pairs, denoted as The spatial granularity contrast losses of all samples are added together as the total spatial contrast loss, denoted as ; Step 6: attention pooling: All support set and query set sample feature maps extracted in step 2 are subjected to attention pooling operation to convert them into support set feature vectors and query set feature vectors; Step 7: query set sample recognition: A prototype of each category is constructed by using the feature vector of the support set sample corresponding to each category, and then the similarity between each query set sample and each category prototype is calculated, and the probability value of the query set sample belonging to each category is calculated by using the Softmax function, and the category with the maximum probability value is taken as the predicted label of the query set sample, and then the cross entropy loss is calculated according to the real label of the query set sample as the small sample classification loss, denoted as ; Step 8: loss optimization: The spatial contrast loss combined with the small sample classification loss to form the total loss of the entire training process, denoted as i.e. ； wherein, and are hyperparameters used to adjust and weights.

2. The small sample target recognition method based on space representation learning and reinforcement according to claim 1, characterized in that, In Step 1, specifically, N classes are randomly selected from the base classes, and K samples are respectively sampled from each class to form a support set S, ; and a number of samples are respectively sampled from each class to form a query set Q, , wherein, represents the i-th support set sample in the support set S, represents the support set sample corresponding to the true label, and N x K represents the number of support set samples in the support set S; represents the i-th query set sample in the query set Q, represents the query set sample corresponding to the true label, and M represents the number of query set samples in the query set Q.

3. The small sample target recognition method based on space representation learning and reinforcement according to claim 2, characterized in that, In step 3, for a support set sample , there are K-1 support set samples and some query set samples of the same class, which form some positive sample pairs with these samples respectively, and form negative sample pairs with the remaining samples of different classes respectively; the remaining samples are the same. In the small sample recognition task constructed in each iteration training, for any sample i, N×K+M-1 positive and negative sample pairs are constructed for it.

4. The small sample target recognition method based on space representation learning and reinforcement according to claim 1, characterized in that, In step 4, use To enhance First, for the feature map A weight vector is obtained using global average pooling. ,Will With feature map The weighted feature map is obtained by multiplying each channel. ,Right now The feature map It contains feature information about the category to which sample i belongs; next, the feature map... The input is sent to a spatial attention module A, and the output is a single-channel attention map of size H×W×1. ,Right now ,in, The parameters in spatial attention module A are represented by H (height), W (width), and C (number of channels); the output attention map is... Feature map of sample j Perform element-wise multiplication to obtain the enhanced feature map. ,Right now ,in This indicates an element-wise multiplication operation.

5. The small sample target recognition method based on space representation learning and reinforcement according to claim 1, characterized in that, In step 5, spatial granularity comparison is performed based on the original feature map and the enhanced feature map of sample i, specifically, the spatial granularity comparison between the original feature map of sample i and the enhanced feature map of sample j based on sample i , both of which have a size of HxWxC, for each feature map, there are HxW pixel points at all spatial positions, and each spatial pixel point p has a C-dimensional feature vector in the corresponding channel direction, which are respectively denoted as and , wherein, represents the feature vector in the corresponding channel direction of the p(th) spatial pixel point on the original feature map of sample i ; represents the feature vector in the corresponding channel direction of the p(th) spatial pixel point on the enhanced feature map of sample j based on sample i ; The similarity between each spatial position of the two feature maps is calculated respectively, and the similarity between each spatial position of the feature map and the feature map is calculated.When calculating the similarity between the spatial position p of the feature map and the feature map , the feature vector of the spatial position p of the feature map and the feature vector of all spatial positions of the feature map are normalized first, and then the similarity between the feature vector of the spatial position p of the feature map and the feature vector of all spatial positions of the feature map is calculated through vector inner product, and the maximum similarity value is taken as the similarity between the spatial position p of the feature map and the feature map , that is, .​ = ； The similarity calculation between the remaining spatial positions and the feature map is the same. Finally, the similarity between all spatial locations and feature maps is summed and averaged to obtain the spatial granularity similarity between the two feature maps, denoted as i.e., = ； Similarly, the spatial granularity similarity between the original feature map of sample j and the enhanced feature map of sample i based on sample j is calculated, denoted as i.e.,​​ = ； in, Represents the original feature map of sample j p ( The feature vectors corresponding to the channel directions of each spatial pixel; This represents the feature map of sample i based on the enhanced feature map of sample j. p ( The feature vectors corresponding to the channel directions of each spatial pixel; The average of the two spatial granularity similarities obtained is taken as the final spatial granularity similarity between sample i and sample j, denoted as i.e., ； The spatial granularity similarity calculation of the remaining sample pairs is the same as steps 4 and 5.

6. The small sample target recognition method based on space representation learning and reinforcement according to claim 5, characterized in that, In step 5, for sample i, the spatial granularity contrast loss function is constructed according to the spatial granularity similarity of its positive and negative sample pairs, denoted as That is, ； wherein, represents the number of samples of the same class as sample i; is an indicator function that takes the value 1 when cond is true and 0 otherwise; is a scalar temperature parameter; finally, the spatial contrastive loss for all samples is summed up as the total spatial contrastive loss, denoted as i.e., 。 7. The small sample target recognition method based on space representation learning and reinforcement according to claim 1, characterized in that, After step 5, the spatial contrast loss is obtained, and then in step 6, the small sample classification loss is calculated using the query set samples. Before calculating the classification loss, the spatial attention mechanism is used in combination with the pooling to convert the feature map into a feature vector. Specifically, the attention pooling operation is used to convert all the support set and query set sample feature maps extracted in step 2 into feature vectors. For sample i, the feature map size is HxWxC. First, the global average pooling is performed to obtain the global feature vector z and copy HxW parts, and then the merged feature map is obtained by merging in the channel direction, denoted as , ; the is input into the attention pooling module , and a series of convolutional layers are output, and the attention map with the same spatial scale as the original feature map , , that is, , wherein represents the convolutional layer in the attention pooling module, represents the convolutional layer parameters; is a logic function; then, the attention map is used to weight and sum each spatial position of the original feature map to obtain the final feature vector, denoted as , , and the calculation formula is ； The process of converting the remaining sample feature maps into feature vectors is the same.

8. A small sample target recognition network based on spatial representation learning and reinforcement, characterized in that, For implementing the small sample target recognition method based on spatial representation learning and reinforcement as claimed in any one of claims 1-7, comprising a feature extraction network, a spatial representation learning module, a spatial representation reinforcement module, and a loss calculation module, The feature extraction network is used to extract the image features of the input sample to obtain a feature map; The spatial representation learning module is used to construct positive and negative sample pairs for the feature map extracted by the feature extraction network, and calculate the spatial contrast loss through feature enhancement mechanism and spatial granularity comparison; The spatial representation reinforcement module is used to convert the feature map extracted by the feature extraction network into a feature vector through attention pooling operation, perform similarity measurement, and output the query set sample prediction label; The loss calculation module is used to calculate the small sample classification loss using the query set sample, and jointly calculate the total loss with the spatial contrast loss.

Images