Noise small sample learning model and method based on analogy reasoning

Abstract

The invention discloses a noise small sample learning method based on analogy reasoning, and relates to the technical field of deep learning and small sample learning, and a noise small sample learning model comprises a trunk feature extraction module, an analogy reasoning module and a task feature comparison learning module. The trunk feature extraction module extracts image features through an encoder based on the noise-doped support set sample data to obtain a category prototype set of all category samples of the support set; obtaining query set sample features based on the query set sample data; the analogy reasoning module obtains analogy knowledge of each corresponding category based on the knowledge base for each category in the training small sample learning task support set; analogy reasoning is carried out through a transform converter, and an enhanced prototype set is obtained; based on the query set sample features and the enhanced prototype set, obtaining category labels of the query set samples; the method can effectively cope with the interference of label noise, has higher robustness compared with an existing method, and improves the performance of small sample learning.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning and few-shot learning, and in particular, to a noise few-shot learning method based on analogical reasoning. Background Art

[0002] Computer vision is one of the important branches in the field of artificial intelligence. As one of the core tasks of computer vision, image classification has broad application prospects. In the era of widespread popularization of machine automation and artificial intelligence, object detection technology has important applications in fields such as autonomous driving, video surveillance, and human-computer interaction. Few-shot learning (FSL) is an important research direction in image classification, mainly solving the problem of model training under the condition of extremely limited sample numbers. Traditional FSL methods usually assume that the labels in the dataset are completely clean. However, in practical applications, label noise is often inevitable. Especially in the process of automatic data collection or manual annotation, noisy labels may seriously affect the training effect of the model.

[0003] Most of the existing few-shot learning methods focus on improving the model's ability to learn from a small number of samples through means such as data augmentation, meta-learning, or metric learning. However, the existing methods do not handle label noise sufficiently. In particular, how to still maintain good performance in an environment of noisy labels remains a difficult problem. Summary of the Invention

[0004] In order to overcome the defect of poor few-shot learning performance in the existing technology under the environment of noisy labels, the present invention proposes a noise few-shot learning method based on analogical reasoning.

[0005] To achieve the above object, the present invention adopts the following technical solutions. A noise few-shot learning method based on analogical reasoning includes:

[0006] S1: Obtain a training few-shot learning task of N classes with K samples Construct a training set for the few-shot learning task; where D support is the support set, and D query is the query set;

[0007] S2: For the training few-shot learning task, add noise to the support set samples to obtain the support set samples doped with noise;

[0008] S3: Construct a noise few-shot learning model, and obtain a trained noise few-shot learning model by learning and training the training few-shot learning task;

[0009] S4: Obtain the task to be tested, divide the task to be tested into a support set and a query set, input the support set data and the query set data into the trained few-shot learning model with noise, and obtain the class labels of the query set samples.

[0010] Preferably, before S1, first divide the data set into D base and D novel , which are used for training and testing respectively, and their corresponding class sets are C base and C novel .

[0011] Preferably, in step S2, the noise includes symmetric label swapping noise and paired label swapping noise.

[0012] Preferably, in step S3, the few-shot learning model with noise includes:

[0013] A backbone feature extraction module, an analogical reasoning module, and a task feature contrast learning module;

[0014] Backbone feature extraction module: Based on the support set sample data doped with noise, extract image features through the encoder f θ to obtain the support set D support The class prototype set P = {w 1 , w 2 ,..., w c ,..., w N} ∈ R d×N ; Obtain query set sample features based on the query set sample data Based on the D base data set and the corresponding classes, establish the knowledge base B = {b c}|c ∈ C base ;

[0015] Among them, i is the sample number, c is the class number, N is the total number of classes, d is the feature dimension, w c is the class prototype corresponding to each class c, b c is the knowledge corresponding to each class c in the knowledge base, f θ (·) represents extracting sample features through the encoder f θ , is the query set sample;

[0016] Analogical reasoning module: For each class c in the support set D support of the training few-shot learning task, obtain the corresponding class analogical knowledge B c from the knowledge base;

[0017] Based on each class analogical knowledge B c and the class prototype wc , perform analogical reasoning through a transformer to obtain an enhanced prototype and a set of enhanced prototypes

[0018]

[0019] Based on the query set sample features and a set of enhanced prototypes obtain the class labels of the query set samples.

[0020] Preferably, the class prototype w corresponding to each class c c is calculated as follows:

[0021]

[0022] The knowledge b corresponding to each class c in the knowledge base c is calculated as follows:

[0023]

[0024] where is the support set sample, is the class label of the support set sample, x i is D base dataset sample, y i is D base dataset sample class label, N c is D base total number of samples of class c in the dataset, is the indicator function, used to find samples with class label c, f θ (.) represents extracting sample features through the encoder f θ Extract sample features.

[0025] Preferably, the task feature contrast learning module is used to train the model, input the class prototype set P and the enhanced prototype set into the task feature extractor, and obtain the merged feature O = {o 1 , o 2 ,..., o c ,...., o N} by merging the forward hidden state and the backward hidden state; obtain the task feature T and the enhanced task feature based on the merged feature through the max pooling layer Based on the task feature T and the enhanced task feature adopt negative sample contrast loss to optimize the model.

[0026] Preferably, the enhanced prototype The calculation formula is as follows:

[0027]

[0028] Wherein, W Q , W K , W V are the query, key, and value projection matrices in the transformer, and Att(.) represents weighted aggregation based on the attention mechanism. represents the matrix transpose of w c , softmax is the normalized exponential function, and d k is the dimension of W K .

[0029] Preferably, the class label of the query set sample has the following calculation formula:

[0030]

[0031] Preferably, the task feature extractor is a bidirectional long short-term memory network, and the update rule is:

[0032]

[0033] Wherein, and are the forward and backward hidden states respectively, and are the forward and backward cell states.

[0034] Preferably, the overall loss function of the few-shot noise learning model has the following calculation formula:

[0035]

[0036] Wherein, τ is the temperature hyperparameter, is the few-shot classification loss of the analogical reasoning module, λ is the weight coefficient, is the negative sample contrast loss of the task feature contrast learning module, y is the corresponding class label; is the enhanced prototype of this category; exp() is the natural exponential function; U - represents the feature set of tasks without overlapping categories with the current task, is the enhanced task feature; T j is the j-th task feature; |D query | represents the number of query set samples.

[0037] The advantages of the present invention are:

[0038] (1) The present invention utilizes an analogical reasoning module. For each category in the support set of the training small-sample learning task, corresponding category analogical knowledge B is obtained based on the knowledge base. c Based on each category analogical knowledge B c and the category prototype w c , analogical reasoning is performed through a transformer to obtain an enhanced prototype. Based on the query set sample features and each enhanced prototype in the enhanced prototype set , the category label of the query set sample is obtained, which can effectively cope with the interference of label noise and has stronger robustness than existing methods, improving the performance of small-sample learning.

[0039] (2) The method of the present invention helps the model understand the relationships between categories through the idea of analogical reasoning, thereby overcoming the influence brought by label noise and improving the accuracy of small-sample learning.

[0040] (3) Through the analogical reasoning module of the present invention, the model can understand the semantic relationships between categories, learn more accurate category prototypes from a small number of labeled samples, and improve the accuracy of few-shot classification.

[0041] (4) The present invention uses a task feature contrast learning module to train the model, thereby better coping with the changes between different tasks and improving the cross-task generalization ability.

[0042] (5) Through the contrast learning mechanism, the present invention contrasts the task features with the enhanced task features, further regularizes the model optimization at the task level, implicitly aligns the original category prototype with the enhanced category prototype, and thus promotes the performance improvement of the small-sample learning task.

[0043] (6) The present invention establishes a knowledge base for storing the semantic relationships between sample data and their categories. Based on the knowledge base, the category analogical knowledge of the support set samples is obtained. By inferring the relationships between new concepts and known concepts, semantic clues are provided for small-sample learning. The transformer of the analogical reasoning module is designed to generate robust and discriminative enhanced category prototypes using analogical relationships. A task-level contrast learning module is introduced to learn the feature distributions between different tasks, improving the learning ability of the model under noisy tasks; improving the generalization ability of the model in small-sample learning tasks under different noise conditions, and at the same time showing competitiveness in traditional small-sample learning tasks, with a wide range of application scenarios, and can be used for target recognition and classification tasks in a noisy environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the architecture diagram of the noise small-sample learning model of the present invention;

[0045] Figure 2This is the flow chart of the method steps of the present invention. Detailed implementation manners

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] As Figure 1-2 shown, the present invention proposes a noise small sample learning method based on analogical reasoning, including:

[0048] Step 1: Obtain the training small sample learning task of N classes and K samples Construct the training set for small sample learning; where D support is the support set, and D query is the query set. Each training small sample learning task is randomly sampled from D base .

[0049] Before step 1, first divide the data set into D base and D novel , which are used for training and testing respectively, and their corresponding class sets are C base and C novel ; and there is no overlap between the classes in the training data set and the test data set. In this embodiment, the ratio of the training data set to the test data set is 8:2.

[0050] Sampling method: For each training small sample learning task , first randomly select N classes from C base , and select K samples for each class to construct the support set samples , then randomly select M additional samples from the N classes as the query set samples , and the samples in the support set and the query set do not repeat. In this embodiment, N = 5, K = 5, M = 15;

[0051] where i is the sample number, x is the sample image data, and y is the sample class label.

[0052] Step 2: For the training small sample learning task, add noise to the support set D support to obtain the support set samples doped with noise;

[0053] The noise includes symmetric label swapping noise and paired label swapping noise. The symmetric label swapping noise randomly selects incorrect samples from the remaining categories in the few-shot learning task; the paired label swapping noise assigns a confused remaining category to each category, and the incorrect samples all come from this confused category. The number of noise samples cannot exceed the number of clean samples in each category, and the number of samples in each category remains unchanged after adding noise.

[0054] Step 3: Build a noisy few-shot learning model. By learning and training the few-shot learning task, obtain a trained noisy few-shot learning model.

[0055] The training process of the noisy few-shot learning model includes: a backbone feature extraction module, an analogical reasoning module, and a task feature contrast learning module.

[0056] Backbone feature extraction module: Based on the support set sample data doped with noise, extract image features through the encoder f θ to obtain the support set D support the class prototype corresponding to each class c in the sample and the prototype set P = {w 1 , w 2 ,..., w c ,..., w N} ∈ R d×N ; Obtain the query set sample features based on the query set sample data where i is the sample number, c is the class number, and f θ (.) represents extracting sample features through the encoder f θ , N is the total number of support set classes, and d is the feature dimension.

[0057] In the backbone feature extraction module, first use a convolutional neural network to build a classifier, pre-train it on D base with the goal of completing classification on C base . After pre-training, remove the classification head in the classifier to obtain the encoder f θ for feature extraction.

[0058] Based on the training set D base sample data and the corresponding classes, establish a knowledge base B = {b c}|c ∈ C base , and the calculation formula for the knowledge b c corresponding to each class c in the knowledge base is:

[0059]

[0060] where i is the sample number, x iis D base the dataset sample, y i is D base the sample class label in D, N c is D base the total number of samples of class c in is the indicator function used to find the samples with class label c, f θ (.) represents extracting the sample features through the encoder f θ

[0061] Analogical reasoning module: For each class c in the support set D of the small-sample learning task during training support find the similar class in C base and obtain the corresponding class analogical knowledge B based on the knowledge base c ; In this embodiment, 5 most relevant types of knowledge in the knowledge base are selected for each class, and finally a set of 5 types of knowledge is formed, namely the class analogical knowledge.

[0062] Based on the class analogical knowledge B c and the class prototype w c , perform analogical reasoning through the transformer to obtain the enhanced prototype as well as the set of enhanced prototypes of N classes

[0063]

[0064] where, W Q , W K , W V are the query, key, and value projection matrices in the transformer represents the matrix transpose of w c , Att represents the weighted aggregation based on the attention mechanism, and the calculation formula is:[[]]

[0065]

[0066] where, softmax is the normalized exponential function, d k is the dimension of W K

[0067] Based on the query set sample features as well as each enhanced prototype in the set of enhanced prototypes obtain the class label of the query set sample The calculation formula is:[[]]

[0068]

[0069] ​​​During this few-shot training stage, the few-shot classification loss is as follows:

[0070]

[0071] where τ is the temperature hyperparameter, is the query set sample image data; y is the corresponding class label; is the augmented prototype for this category; exp() is the natural exponential function; |D query | represents the number of query set samples; in this embodiment, τ is 0.1.

[0072] Task feature contrastive learning module: used to train the model. In the present invention, a bidirectional long short-term memory network (LSTM) is used as the task feature extractor, and the number of LSTM layers corresponds to the total number N of support set categories. The category prototype set P and the augmented prototype set are respectively input into the bidirectional LSTM, and the forward hidden state and the backward hidden state are merged to obtain the merged feature of the bidirectional LSTM as the output O = {o 1 , o 2 ,..., o c ,...., o N}, where Based on the output O of the bidirectional LSTM, through max pooling aggregation, the task feature T and the augmented task feature are obtained. In this embodiment, the dimension of the bidirectional LSTM is 256, so the dimension of the task feature T is 512.

[0073] For the bidirectional LSTM, the calculation process of each layer of the forward and the backward is as follows:

[0074]

[0075] where w c is the category prototype corresponding to category c; and are the forward and backward cell states; in particular, is a randomly initialized vector.

[0076] Based on the task feature T and the augmented task feature the negative sample contrast loss is used to optimize the model.

[0077] Through the contrastive learning mechanism, the task feature T and the augmented task feature Make a comparison, implicitly align the original class prototypes and the enhanced prototypes, narrow the distance between similar tasks, and widen the distance between dissimilar tasks, so as to further regularize the model optimization at the task level, thereby promoting the performance improvement of few-shot tasks.

[0078] To perform task-level contrastive learning, we use the negative sample contrast loss In each training step, through a few-shot task, the current task feature T is stored in a queue for subsequent contrastive learning to calculate the negative sample contrast loss.

[0079] Since each few-shot task is randomly sampled, the tasks in the task queue may have a high similarity with the current task. In this case, it is not appropriate to use these tasks as negative samples for contrastive learning. To avoid this situation, the present invention proposes a negative sample selection strategy, by selecting tasks that are completely different from the current task (i.e., no class overlap) from the queue as negative samples, thereby reducing the interference of highly relevant tasks on model optimization. Specifically, the negative sample contrast loss is defined as:

[0080]

[0081] where U - represents the feature set of tasks without overlapping classes with the current task , τ is the temperature hyperparameter, is the enhanced task feature; T j is the j-th task feature; in this embodiment, the storage length of U is 256 and τ is 0.1.

[0082] Using the few-shot task training set sampled in D base to optimize the model, the total loss function includes the few-shot classification loss and the negative sample contrast loss. Based on the overall loss function where λ is the weight coefficient, the model is trained to balance the influence of the two. When the number of training iterations reaches the set number or the overall loss function converges, the training stops, thus obtaining a trained few-shot learning model with noise.

[0083] In this embodiment, the Adam optimizer is adopted, the initial learning rate is 1e-3, and the decay method is that the learning rate is halved every 20k tasks during training. The training process has a total of 200k tasks.

[0084] Step 4: Obtain the task to be tested, divide the task to be tested into a support set and a query set, input the support set data and the query set data into the trained few-shot learning model with noise, and obtain the class labels of the query set samples; including:

[0085] The task to be tested is a small sample testing task of N classes with K samples per class Each small sample testing task is randomly sampled from D novel and obtained.

[0086] Sampling method: For each small sample testing task First, randomly select N classes from C novel and select K samples from each class to construct a support set Then, randomly select an additional M samples from the N classes as a query set and the samples in the support set and the query set are not repeated. In this embodiment, N = 5, K = 5, and M = 15;

[0087] Use the encoder f of the trained small sample learning model with noise θ , and according to the support set D of the task to be tested support calculate and obtain the prototype set P = {w 1 , w 2 ,..., w c ,..., w N}; According to the query set D of the task to be tested query calculate and obtain the query set sample features

[0088] Use the analogical reasoning module. For each class in the query set of the small sample testing task, find the most relevant class in C base and obtain the corresponding class analogical knowledge based on the knowledge base; Based on the class analogical knowledge and the prototype set P, through the transformer, calculate and obtain the enhanced prototype set

[0089] Based on the query set sample features and the enhanced prototypes in the task to be tested, obtain the class labels of the query set samples The calculation formula is:

[0090]

[0091] where i is the query set sample number.

[0092] Few-shot learning, also known as small-sample learning, aims to solve the problem of data scarcity. In practical applications, obtaining a large amount of labeled data is often costly and time-consuming. Therefore, few-shot learning has important practical significance. It requires the model to effectively learn from limited samples and possess the ability to quickly adapt to and generalize new tasks. At the same time, in the case of few samples and the presence of noise, the present invention provides a noise few-shot learning method based on analogical reasoning, which improves the generalization ability of the model in few-shot learning tasks under different noise conditions, and at the same time shows competitiveness in traditional few-shot learning tasks, significantly reducing data requirements and improving data utilization efficiency. It has a wide range of application scenarios and can be used for target recognition and classification tasks in noisy environments, scenarios where the cost of labeled data is high and samples are scarce, such as medical image analysis, remote sensing image classification, rare species identification, industrial defect detection, etc.

[0093] To verify the effectiveness of the method of the present invention, in this embodiment, the above method is applied to the few-shot learning classification task with noise, and performance comparison is carried out with the current mainstream noise few-shot learning algorithms on the Mini-ImageNet dataset. The performance comparison of each method in the few-shot learning task is shown in Table 1, giving the average accuracy calculated on 600 test few-shot tasks and the 95% confidence interval.

[0094] Table 1 Performance comparison of each method in the few-shot learning task

[0095]

[0096] As can be seen from Table 1, compared with the mainstream noise few-shot learning algorithms, the method of the present invention achieves the highest classification accuracy on the Mini-ImageNet dataset under each noise setting, and the advantage of the method of the present invention becomes more obvious as the noise ratio increases. This means that this method can better utilize a small amount of data for few-shot learning and has better robustness to noise.

[0097] Of course, for those skilled in the art, the present invention is not limited to the details of the above exemplary embodiments, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

[0098] In addition, it should be understood that although this specification is described in terms of embodiments, not every embodiment contains only an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0099] The technologies, shapes, and structures not detailedly described in the present invention are all well-known technologies.

Claims

1. A noisy small sample learning method based on analogical reasoning, characterized in that: include: S1: Get N types of K samples for training small sample learning task Construct a training set for small sample learning tasks; where D support is the support set, D query is the query set; S2: For the training small sample learning task, noise is added to the support set samples to obtain the support set samples doped with noise; S3: Construct a noise small sample learning model, and obtain a trained noise small sample learning model by training the training small sample learning task; S4: Obtain the task to be tested, divide the task to be tested into a support set and a query set, input the support set data and the query set data into the trained noise small sample learning model, and obtain the category label of the query set sample.

2. A noisy small sample learning method based on analogical reasoning as claimed in claim 1, characterized in that: Before S1, the dataset is first divided into D base and D novel , respectively, for training and testing, and their corresponding category sets are C base and C novel .

3. The noisy small sample learning method based on analogical reasoning as claimed in claim 1, characterized in that: In step S2, the noise includes symmetric label exchange noise and paired label exchange noise.

4. The noisy small sample learning method based on analogical reasoning as claimed in claim 2, characterized in that: In step S3, the noise small sample learning model includes: Backbone feature extraction module, analogy reasoning module and task feature comparison learning module; Backbone feature extraction module: Based on the support set sample data doped with noise, through the encoder f θ Extract image features and get support set D support The category prototype set P of all category samples is {w1,w2,...,w c ,...,w N }∈R d×N ; Get query set sample features based on query set sample data Based on D base Data sets and corresponding categories, establish knowledge base B = {b c }|c∈C base ; Among them, i is the sample number, c is the category number, N is the total number of categories, d is the feature dimension, and w is the c For each category c, the corresponding category prototype, b c is the knowledge corresponding to each category c in the knowledge base, f θ (·) indicates that the encoder f θ Extract sample features, is a sample of the query set; Analogy reasoning module: for training small sample learning tasks support set D support For each category c in the , obtain the corresponding category analogy knowledge B based on the knowledge base c ; Based on each category analogy knowledge B c and the class prototype w c , perform analogical reasoning through the transformer to obtain an enhanced prototype And enhanced prototype collection Based on query set sample features And enhanced prototype collection Get the class labels of the query set samples.

5. The noisy small sample learning method based on analogical reasoning as claimed in claim 4, characterized in that: The category prototype w corresponding to each category c c The calculation formula is: Each category c in the knowledge base corresponds to knowledge b c The calculation formula is: in, is the support set sample, is the class label of the support set samples, x i D base Dataset sample, y i D base Sample category labels in the dataset, N c D base The total number of samples of category c in the dataset, is the indicator function, Used to find samples with category label c, f θ (.) indicates that the encoder f θ Extract sample features.

6. A noisy small sample learning method based on analogical reasoning as claimed in claim 4, characterized in that: The task feature contrast learning module is used to train the model by comparing the category prototype set P and Enhanced Prototype Collection Input to the task feature extractor, and obtain the merged feature O = {o1, o2, ..., o c ,....,o N }; Based on the merged features, the task features T and enhanced task features are obtained through the maximum pooling layer Based on task feature T and enhanced task feature Using negative sample contrast loss to optimize the model.

7. A noisy small sample learning method based on analogical reasoning as claimed in claim 4, characterized in that In, Enhanced Prototype The calculation formula is: Among them, W Q , W K , W V are the query, key, and value projection matrices in the transformer. Att(.) represents the weighted aggregation based on the attention mechanism. Represents w c The matrix transpose of , softmax is the normalized exponential function, d k W K Dimension.

8. The noisy small sample learning method based on analogical reasoning as claimed in claim 4, characterized in that: The category labels of the query set samples The calculation formula is:

9. The noisy small sample learning method based on analogical reasoning as claimed in claim 6, characterized in that: The task feature extractor is a bidirectional long short-term memory network, and the update rule is: in, and are the forward and reverse hidden states, respectively. and Forward and reverse cell states.

10. The noisy small sample learning method based on analogical reasoning as claimed in claim 4, characterized in that: The overall loss function of the noisy small-shot learning model is The calculation formula is: Where τ is the temperature hyperparameter, is the small sample classification loss of the analogy reasoning module, λ is the weight coefficient, is the negative sample contrast loss of the task feature contrast learning module, and y is The corresponding category label; This is the enhanced prototype of this category; exp() is the natural exponential function; U - Represents the feature set of tasks that have no overlap with the current task. To enhance the task characteristics; T j is the jth task feature; |D query | represents the number of query set samples.

Images