A small sample learning method and system based on prototype completion and dynamic loss adjustment

Abstract

The application discloses a small sample learning method and system based on prototype completion and dynamic loss adjustment, and the method comprises the following steps: step one: integrating the information and labels of an original support set and an original query set to obtain new prototypes, and adding the new prototypes to the original support set to form a new support set; step two: using a feature map reconstruction network to obtain a reconstructed query set according to the new support set; step three: calculating the loss of the original query set and the reconstructed query set based on a constructed dynamic loss function, and training the feature reconstruction network by minimizing the loss; and step four: inputting a to-be-classified image data set into the trained feature reconstruction network to obtain an image classification result. By adopting the two strategies of prototype completion and dynamic adjustment of a loss function, the application significantly improves the performance of a small sample classification task.

Description

Technical Field

[0001] This invention relates to the field of few-shot learning, and more particularly to a few-shot learning method and system based on prototype completion and dynamic loss adjustment. Background Technology

[0002] The unique feature of deep learning lies in its ability to autonomously learn features from massive amounts of data, rather than relying on manual design. This high-quality feature learning significantly improves the performance of recognition systems. However, in practical applications in fields such as medicine, military, and finance, sufficient labeled training samples are difficult to obtain due to factors such as privacy, security, or data annotation costs. This scarcity of data limits the further expansion of deep learning in the field of image classification. To address this, few-shot learning techniques have emerged, aiming to extract new conceptual understandings from a small number of samples, relying on rich basic category data to identify new categories under limited labeled samples.

[0003] One approach to solving the few-shot learning problem is to construct a feature space where features for samples of different classes can be concentrated within classes and diverged between classes. This feature space is typically represented by a deep neural network, which infers the class of a test sample by leveraging the similarity between the test and training samples and passing the label information from the training samples. Prototype networks classify new samples by directly comparing them to the prototypes of each class, rather than calculating the similarity between the new sample and the support set samples, where the prototype of each class is a single vector representation of that class in the embedding space. Furthermore, data-augmented few-shot image classification strategies increase the number of training samples by creating new auxiliary data to expand the original dataset.

[0004] The performance of prototype networks is highly dependent on the quality of the feature space. When faced with complex or unevenly distributed data, the model may struggle to capture sufficient class differences, leading to decreased classification accuracy. Furthermore, it may require more samples to form an accurate prototype when handling new classes. Summary of the Invention

[0005] To at least partially address the challenge of data scarcity in few-shot learning, this invention provides a few-shot learning method and system based on prototype completion and dynamic loss function adjustment. Through these two strategies, prototype completion and dynamic loss function adjustment, the performance of few-shot classification tasks is effectively improved. Prototype completion enhances the representational power of the support set, enabling more sensitive identification of subtle class differences. Secondly, the introduction of a dynamic loss function adjustment mechanism increases adaptability and accuracy for different few-shot scenarios. Finally, comparative and ablation experiments further demonstrate the crucial role of prototype completion and dynamic loss function adjustment in improving performance.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] The first aspect of this invention proposes a few-shot learning method based on prototype completion and dynamic loss adjustment, comprising the following steps:

[0008] Step 1: Integrate the information from the original support set and the original query set, generate a new prototype through label propagation, and add the new prototype to the original support set to form a new support set, which enriches the support set and increases the flexibility and accuracy of the classification task.

[0009] Step 2: Use feature maps to reconstruct the network and obtain the reconstructed query set based on the new support set. This helps to preserve spatial details and improves generalization ability and classification accuracy.

[0010] Step 3: Calculate the loss of the original query set and the reconstructed query set based on the constructed dynamic loss function, and train the feature reconstruction network by minimizing the loss. The dynamic loss function ensures that it is not overconfident in the majority class samples and pays more attention to the minority class and difficult-to-classify samples, thereby improving the overall robustness and performance.

[0011] Step 4: Input the dataset of images to be classified into the trained feature reconstruction network to obtain the image classification results.

[0012] Furthermore, step one specifically includes:

[0013] Calculate the average feature offset between the original support set and the original query set, and correct the features of the original query set based on the average feature offset, so that the corrected query set features are closer to the feature distribution of the support set.

[0014] Calculate the similarity between the original support set features and the corrected query set features and the prototype, and convert both similarity values ​​into probability distributions to facilitate the calculation of the probability distributions of the reweighted support set and query set based on the probability distributions.

[0015] The probability distributions of the original support set and the corrected query set are reweighted to obtain the reweighted probability distributions of the support set and query set, which facilitates updating the prototype.

[0016] The probability distributions of the reweighted support set and query set are applied to their respective features to update the prototype. The updated prototype is then added to the support set as a new sample, resulting in a new support set. This approach improves the flexibility and accuracy of classification tasks.

[0017] Furthermore, the corrected query set features are calculated according to the following formula:

[0018]

[0019]

[0020] F q ′=F q +Δ

[0021] Among them, Q i Let S be the feature matrix of the i-th query sample. i Let n be the feature matrix of the i-th supporting sample. support n is the number of categories in the original support set. query The number of categories in the original query set. and F represents the average values ​​of the features in the original support set and the features in the original query set, respectively. q ′ is the corrected query set feature, F q Δ represents the features of the original query set, and Δ represents the difference between the average of the features of the original support set and the features of the original query set.

[0022] The probability distributions of the original support set and the corrected query set are calculated using the following formula:

[0023] P s =exp(cos(F) s ,P))

[0024] P q =exp(cos(F) q ′,P))

[0025] Among them, P s and P q These are the probability distributions of the original support set and the corrected query set relative to the prototype, respectively; cos is the cosine similarity function; and exp is the exponential function.

[0026] Calculate the probability distribution of the reweighted support set and query set using the following formula:

[0027]

[0028] Where N is the normalized vector, n s and n q These are the number of samples in the original support set and the corrected query set, respectively, L si and L qi These are the labels of the i-th sample in the original support set and the corrected query set, respectively, where j is the number of categories, and P... si and P qi These are the probability distributions of the original support set and the corrected query set relative to the prototype, respectively. One-hot encoding is used for one-hot coding. W... q W represents the probability distribution of the reweighted query set. s The probability distribution of the reweighted support set;

[0029] The new support set is calculated using the following formula:

[0030]

[0031] S′=S+P

[0032] Where P is the updated prototype, S is the original support set, and S′ is the new support set.

[0033] Furthermore, in step three, the dynamic loss function construction process specifically includes:

[0034] A weight adjustment matrix is ​​initialized based on one-hot encoding, and the output of the feature reconstruction network is processed by softmax to obtain the probability matrix.

[0035] The predicted probability of each sample to the true label is calculated based on the probability matrix, and all samples are traversed to update the weight adjustment matrix. If the predicted probability of the true label of a sample is lower than the set threshold, the weight of the category corresponding to the true label of the sample is increased, and the weight of other categories is decreased to obtain the new weight of the sample. The overall generalization performance is improved through dynamic weights.

[0036] The loss for each sample is calculated, and the weighted losses of all samples are summed using a weight adjustment matrix to obtain the total loss, which allows for more attention to samples that are poorly predicted or difficult to predict.

[0037] Furthermore, the true label probability is expressed by the following formula:

[0038]

[0039] Pt i =P i,j

[0040] Where C is the total number of categories, x i,j P is the output of the feature reconstruction network for the i-th sample in the j-th category. i,j Let Pt be the probability matrix of the i-th sample in the j-th class. i Let be the predicted probability of the true label of the i-th sample;

[0041] The weights of the samples are expressed by the following formula:

[0042]

[0043] Among them, w i Let be the weight corresponding to the i-th sample, C be the total number of categories, and eps be a very small constant.

[0044] The loss for each sample can be expressed using the following formula:

[0045]

[0046] Where γ is the parameter controlling the weights of positive and negative samples, L i Let Pt be the loss for the i-th sample, where C is the total number of classes. i Let P be the predicted probability of the true label of the i-th sample. i,j Let be the sample probability matrix of the i-th sample in the j-th category.

[0047] A second aspect of this invention proposes a few-shot learning system based on prototype completion and dynamic loss adjustment, comprising:

[0048] The new prototype generation module is used to integrate information from the original support set and the original query set, generate new prototypes through label propagation, and add the new prototypes to the original support set to form a new support set, which enriches the support set and increases the flexibility and accuracy of the classification task.

[0049] The feature map reconstruction module is used to reconstruct the query set based on the new support set using the feature map reconstruction network, which helps to preserve spatial details and improves generalization ability and classification accuracy.

[0050] The training module is used to calculate the loss of the original query set and the reconstructed query set based on the constructed dynamic loss function. The feature reconstruction network is trained by minimizing the loss. The dynamic loss function ensures that it is not overconfident in the majority class samples and pays more attention to the minority class and difficult-to-classify samples, thereby improving the overall robustness and performance.

[0051] The classification module is used to input the dataset of images to be classified into a trained feature reconstruction network to obtain the image classification results.

[0052] A third aspect of the present invention provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a few-shot learning method based on prototype completion and dynamic loss adjustment as described in the first aspect above.

[0053] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform a few-shot learning method based on prototype completion and dynamic loss adjustment as described in the first aspect above.

[0054] The beneficial effects of this invention are:

[0055] (1) This invention uses nearest neighbor samples between support samples and query samples to expand the support samples of the computational prototype. The prototype is updated by weighted summation based on similarity, and the new prototype is used as a support sample for data augmentation. Simultaneously, a dynamic adjustment strategy for the loss function is applied to make the feature reconstruction network focus on those difficult-to-classify samples, thereby significantly improving the accuracy of small-sample classification tasks. By meticulously reducing intra-class and cross-class bias, accurate reconstruction and adjustment of the prototype are achieved, resulting in leading performance on multiple standard small-sample learning datasets.

[0056] (2) By introducing a prototype completion mechanism, this invention captures key features within a class and effectively generates more representative and accurate class prototypes, thereby enhancing the understanding and representation of support set data.

[0057] (3) This invention utilizes the features after prototype completion to perform data augmentation in the feature map reconstruction module, effectively combining the techniques of prototype completion and feature reconstruction, enriching the training dataset, and significantly improving the adaptability and generalization ability to unknown categories by optimizing feature representation.

[0058] (4) The present invention adopts a dynamic loss function adjustment strategy, which can pay more attention to samples that are difficult to classify, and ensure that a few-shot learning system based on prototype completion and dynamic loss adjustment can adaptively adjust during the learning process, and realize personalized loss weight adjustment for samples of different difficulties, thereby improving classification accuracy while maintaining the robustness of a few-shot learning system based on prototype completion and dynamic loss adjustment. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a few-shot learning method based on prototype completion and dynamic loss adjustment, provided in an embodiment of the present invention.

[0060] Figure 2 This is a structural diagram of a feature reconstruction network based on a few-shot learning method using prototype completion and dynamic loss adjustment, provided in an embodiment of the present invention.

[0061] Figure 3 This is a structural diagram of a novel prototype generation module for a few-shot learning system based on prototype completion and dynamic loss adjustment, provided in an embodiment of the present invention.

[0062] Figure 4 The graph shows the ablation experiment results on a fine-grained dataset provided by an embodiment of the present invention, based on a few-shot learning method with prototype completion and dynamic loss adjustment.

[0063] Figure 5 The graph shows the ablation experiment results on a general dataset for a few-shot learning method based on prototype completion and dynamic loss adjustment, which is provided in the embodiments of the present invention.

[0064] Figure 6 The training curves on a fine-grained few-sample benchmark dataset for a few-sample learning method based on prototype completion and dynamic loss adjustment, provided in an embodiment of the present invention.

[0065] Figure 7 The training curve of a few-shot learning method based on prototype completion and dynamic loss adjustment provided in this embodiment of the invention on a general few-shot benchmark dataset.

[0066] Figure 8 This is an architecture diagram of a few-shot learning system based on prototype completion and dynamic loss adjustment, provided for an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0068] Example 1

[0069] like Figure 1 As shown, a few-shot learning method based on prototype completion and dynamic loss adjustment includes the following steps:

[0070] S101: Integrate the information from the original support set and the original query set, generate a new prototype through label propagation, and add the new prototype to the original support set to form a new support set.

[0071] S102: Reconstruct the network using feature maps to obtain the reconstructed query set based on the new support set.

[0072] S103: Calculate the loss between the original query set and the reconstructed query set based on the constructed dynamic loss function, and train the feature reconstruction network by minimizing the loss.

[0073] S104: Input the dataset of images to be classified into the trained feature reconstruction network to obtain the image classification results.

[0074] Few-shot classification is an extremely challenging task where there are only a handful of samples for each new class. Traditional classifiers often suffer from overfitting when faced with such tasks, leading to a decrease in classification accuracy.

[0075] like Figure 2As shown, this invention employs a data augmentation strategy to expand the sample size of the support set. The core method involves selecting nearest-neighbor samples from the original support set within the original query set to expand the sample base upon which the prototype is based, and then fine-tuning the prototype based on the similarity between samples. This corrected prototype is considered a new sample, thereby substantially increasing the data size of the support set.

[0076] Furthermore, to further improve the performance of the feature reconstruction network, this invention designs a dynamic loss function to enhance positive samples. Specifically, it reduces the loss for positive samples while enhancing the loss for negative samples, thereby making the feature reconstruction network focus more on samples that are difficult to classify. This method allows the feature reconstruction network to pay more attention to those challenging samples, thus improving the classification accuracy of the feature reconstruction network.

[0077] Example 2

[0078] Based on the above embodiments, this invention provides a specific process for constructing a new support set, which specifically includes:

[0079] Prototype networks are a method for few-shot learning. The core idea is to learn a space where prototypes of the same class can represent the entire category. During classification, it calculates the distance between the input sample and the prototypes of each class, assigning the sample to the nearest prototype class. The basic formula for prototype networks typically involves calculating the distance between the input sample and the prototype of each class, and then assigning the sample to the nearest prototype. Mathematically, this is expressed by the following formula:

[0080]

[0081] Where f(x) is the feature representation of the input sample x, c k y is the prototype of the k-th category, d is the distance function, usually Euclidean distance or cosine distance, p(y=k│x) represents the probability that sample x belongs to category k, and exp is the exponential function.

[0082] Prototype networks utilize distance metrics, such as cosine similarity and Euclidean distance, to assign query set samples to the nearest prototype. These metrics are concise, intuitive, and provide powerful and effective classification capabilities. They can effectively evaluate the similarity between samples in various application scenarios, promoting the construction of robust classifiers.

[0083] Prototype networks can perform poorly when dealing with uneven datasets, especially when certain class representations in the support set are insufficient, potentially causing the model to fail to capture features from all classes adequately. Prototype completion addresses this issue by adjusting or adding prototypes, enhancing the feature reconstruction network's ability to capture subtle class differences with limited samples, thereby improving the network's generalization and accuracy.

[0084] Unlike previous methods that directly use the completed prototypes for comparative classification, this invention adds these newly generated prototypes as samples to the support set and further uses them to train the feature reconstruction network. This not only increases the diversity of training data, but also improves feature representation. By combining the advantages of prototype completion and feature reconstruction, it helps to improve the generalization ability to unseen categories, thereby achieving more accurate classification results in the domain of few-shot learning.

[0085] As one possible implementation, this invention uses a Convolutional Spatial Propagation Network (CSPN) for prototype completion. First, it integrates information and label propagation from the original support set and the original query set. Then, it calculates the probability distribution from sample to prototype using cosine similarity. Finally, it updates the prototype using feature translation and probability weighting. Figure 3 As shown, the new prototype after prototype completion is not directly compared with the query set as before, but is treated as a new sample and added to the original support set to obtain a new support set. Enriching the original support set with new prototypes enhances the generalization ability of the feature reconstruction network, providing more representative prototypes for each category, thereby improving the accuracy and overall performance in identifying less common categories. By adding new prototypes to the support set, the feature reconstruction network can better adapt to new categories, improving the flexibility and accuracy of the classification task.

[0086] S201: Calculate the average feature offset between the original support set and the original query set, and correct the features of the original query set based on the average feature offset.

[0087] Specifically, the average feature offset between the original support set and the original query set is calculated. The features of the original query set are then corrected based on this average feature offset. The purpose is to adjust the features of the original query set to be closer to the feature distribution of the original support set, as expressed by the following formula:

[0088]

[0089] F q ′=F q +Δ (5)

[0090] Among them, Q i Let S be the feature matrix of the i-th query sample. iLet n be the feature matrix of the i-th supporting sample. support n is the number of categories in the original support set. query The number of categories in the original query set. and F represents the average values ​​of the features in the original support set and the features in the original query set, respectively. q ′ is the corrected query set feature, F q Δ represents the features of the original query set, and Δ represents the difference between the average of the features of the original support set and the features of the original query set.

[0091] S202: Calculate the similarity between the original support set features and the corrected query set features and the prototype, and convert the similarity of both into a probability distribution.

[0092] Specifically, cosine similarity is used to calculate the similarity between the original support set features and the corrected query set features and the prototype, and then an exponential function is applied to convert it into a probability distribution, expressed by the following formula:

[0093] P s =exo(cos(F s ,P)) (6)

[0094] P q =exo(cos(F q ′,P)) (7)

[0095] Among them, P s and P q These are the probability distributions of the original support set and the corrected query set relative to the prototype, respectively. cos is the cosine similarity function, and exp is the exponential function.

[0096] S203: Reweight the probability distributions of the original support set and the corrected query set to obtain the reweighted probability distributions of the support set and the query set.

[0097] Specifically, the prototype is updated by combining label information and probability distribution, using one-hot encoded labels for label propagation. The probability distributions of the original support set and the corrected query set are reweighted so that the total probability of each category is 1, as expressed by the following formula:

[0098]

[0099] Where N is the normalized vector, n s and n q These are the number of samples in the original support set and the corrected query set, respectively, L si and L qi These are the labels of the i-th sample in the original support set and the corrected query set, respectively, where j is the number of categories, and P... si and Pqi These are the probability distributions of the original support set and the corrected query set relative to the prototype, respectively. One-hot encoding is used for one-hot coding. W... q W represents the probability distribution of the reweighted query set. s The probability distribution of the reweighted support set.

[0100] S204: Apply the reweighted probability distributions of the support set and query set to their respective features to update the prototype. Add the updated prototype as a new sample to the support set to obtain a new support set.

[0101] Specifically, the prototype is updated using the reweighted probability distribution. The reweighted probability distributions of the support set and query set are applied to their respective features, and then these weighted features are summed to update the prototype. Finally, the updated prototype is added to the support set as a new sample, as expressed by the following formula:

[0102]

[0103] S′=S+P (13)

[0104] Where P is the updated prototype, S is the original support set, and S′ is the new support set.

[0105] Example 3

[0106] Based on the above embodiments, this invention provides a process for constructing a feature map reconstruction network, specifically including:

[0107] S301: Regress from support features to original query set features to predict the category of the original query set samples, generate corresponding feature maps for a group of images belonging to the same category, and aggregate the component feature vectors of each location and image into a support feature pool.

[0108] Specifically, given a set of support images from k samples across n classes and their corresponding class labels, predict a class label for a single input query image. For each class c, aggregate all features from the k support images into a single support feature matrix S. c middle.

[0109] S302: Define class membership by reconstructing each position in the feature map as a weighted sum of supporting features.

[0110] Specifically, the feature map reconstruction network attempts to find a variable matrix W such that WS c ≈Q to reconstruct the feature map of the query image. Q is used as S. c The weighted sum of the Bank of China's calculations is expressed by the following formula:

[0111]

[0112] Where ||·|| represents the Frobenius norm, S c W is a matrix composed of features extracted from the supporting images, and W is the reconstruction weight matrix. It is an approximate matrix. It is S c The transpose of , where I is the identity matrix, and λ is the regularization term in ridge regression, used to ensure stability and solvability when solving linear least squares problems. It prevents overfitting by introducing a penalty term during optimization and maintains stability when dealing with over- or under-constrained linear systems. In cases of small samples or sparse data, λ helps balance the reconstruction accuracy of feature maps with the generalization ability of the feature reconstruction network.

[0113] S303: For each query image, compute the best-fit feature map reconstruction for each category, and the closest candidate category is predicted as the category of the query image.

[0114] Specifically, the reconstructed feature mapping of category c Calculate according to the following formula:

[0115]

[0116] in, For the reconstructed feature mapping of category c, S is an approximate matrix. c The matrix is ​​composed of features extracted from the supporting images, and ρ is a learned recalibration term used to fine-tune the reconstruction process. Its function is to adjust the size of the reconstruction weight matrix W, thereby changing the reconstructed feature mapping. The size of ρ. By adjusting ρ, the training data can be better adapted, optimizing the accuracy of the reconstruction. In the context of FRN, ρ allows for penalizing large weights while maintaining the relative size of the reconstruction weights, thus avoiding overfitting and improving generalization ability.

[0117] Example 4

[0118] Based on the above embodiments, this invention provides a process for constructing a dynamic loss function, specifically including:

[0119] This invention employs a weighted smooth focus loss function (WSFL) to guide the feature reconstruction network to prioritize learning from difficult-to-classify positive samples, thereby improving its performance in small-sample scenarios. Class imbalance is addressed by adjusting the weights for each sample's class in the training batch. More significant weight adjustments are made to samples with low prediction confidence (i.e., class probabilities below a predefined threshold θ), thus optimizing the network's generalization performance. This approach ensures that the feature reconstruction network does not overestimate the majority class and focuses more on the minority class and difficult-to-classify samples, thereby improving its overall robustness and performance.

[0120] S401: Initialize a weight adjustment matrix based on one-hot encoding, and process the output of the feature reconstruction network through softmax to obtain the probability matrix.

[0121] Specifically, a weight adjustment matrix W is first initialized based on one-hot encoding, expressed by the following formula:

[0122]

[0123] Among them, L s The label of the new support set is 1. If the label of the i-th new support set is j, the value of the corresponding position in the W matrix is ​​set to 1. Otherwise, it is 0.

[0124] The probability matrix obtained after the output of the feature reconstruction network is processed by softmax is shown below:

[0125]

[0126] Where C is the total number of categories, x i,j It is the feature reconstruction network output of the i-th sample in the j-th class (i.e., the distance between the original query set and the reconstructed query set is calculated using Euclidean distance, with the distance taking a negative value), P i,j Let be the sample probability matrix of the i-th sample in the j-th category.

[0127] S402: Calculate the predicted probability of each sample to the true label based on the probability matrix, and iterate through all samples to update the weight adjustment matrix. If the predicted probability of the true label of a sample is lower than the set threshold, increase the weight of the category corresponding to the true label of the sample and decrease the weight of other categories to obtain the new weight of the sample.

[0128] Specifically, the probability of the true label Pt corresponding to the sample is: Pt i =P i,j Pt i Let be the predicted probability of the true label of the i-th sample.

[0129] For each sample, the weights are adjusted based on its value. If the predicted probability of a sample is lower than a set threshold θ, it means that the feature reconstruction network is not confident enough in predicting that sample, and the weights of the corresponding class need to be increased, while the weights of other classes need to be decreased accordingly. θ is a preset threshold (usually 1 / N), and eps is a small constant used to adjust the weights, which can be achieved using the following formula:

[0130]

[0131] Among them, w i Let be the weight corresponding to the i-th sample, C be the total number of categories, and eps be a very small constant.

[0132] S403: Calculate the loss for each sample and sum the weighted losses of all samples using the weight adjustment matrix to obtain the total loss.

[0133] Specifically, the loss L for each sample is calculated using a weighted log probability. This reduces the focus on samples correctly classified by the feature reconstruction network, allowing the network to concentrate more on samples that are difficult to classify. The calculation formula is shown below:

[0134]

[0135] Where γ is the parameter controlling the weights of positive and negative samples, L i Let Pt be the loss for the i-th sample, where C is the total number of classes. i Let P be the predicted probability of the true label of the i-th sample. i,j Let be the sample probability matrix of the i-th sample in the j-th category.

[0136] Next, the loss for each sample is multiplied by its corresponding weight. Through this weighting, the feature reconstruction network pays more attention to samples that are poorly predicted or difficult to predict. Finally, the weighted losses of all samples are summed and divided by the sample size N to obtain the average loss for the entire batch, as shown in the formula below:

[0137]

[0138] Where Loss is the average loss for the entire batch, N is the sample size, and L is the average loss for the entire batch. i Let be the loss for the i-th sample.

[0139] WSFL makes feature reconstruction networks more efficient at identifying hard-to-classify samples. This unique loss function strategy ensures that feature reconstruction networks achieve more accurate classification results when dealing with variable datasets. By improving the ability to identify hard-to-classify samples, WSFL not only enhances the generalization ability of feature reconstruction networks but also improves their adaptability and accuracy in various scenarios, especially when facing challenges such as limited sample sizes or imbalanced class distributions.

[0140] Example 5

[0141] Based on the above embodiments, this invention provides a testing process for feature reconstruction networks.

[0142] To comprehensively evaluate the performance of the feature reconstruction network, this invention was experimentally validated on two key benchmarks: fine-grained few-shot and general few-shot benchmarks. Experimental results show that the feature reconstruction network exhibits superior performance in several key areas. It not only achieves significant improvements in accuracy when handling fine-grained and general few-shot learning tasks, but also demonstrates excellent generalization ability and stability.

[0143] In fine-grained, small-sample benchmark testing, this invention utilizes the CUB dataset, which consists of approximately 11,788 images distributed across 200 bird categories, averaging about 60 images per category. Each image in the CUB dataset is equipped with rich annotation information, covering bounding boxes, location details, and attribute labels, providing abundant visual and semantic information for fine-grained image analysis.

[0144] Following the experimental setup of previous studies, the CUB dataset was randomly divided into 100 training classes, 50 validation classes, and 50 evaluation classes. This class division strategy aims to ensure the comprehensiveness and effectiveness of the feature reconstruction network training and evaluation. To ensure the comparability of experimental results, experiments were conducted in two different settings to address different data preprocessing methods previously used in this benchmark: one using the original images directly as input, and the other cropping each image based on manually labeled bounding boxes. This dual experimental design comprehensively evaluates and validates the performance of the proposed feature reconstruction network in fine-grained image classification tasks, particularly its adaptability and robustness to diverse data preprocessing strategies.

[0145] This invention also incorporates the Aircraft dataset to further validate the generalization ability and adaptability of the feature reconstruction network. The Aircraft dataset consists of approximately 10,000 images, covering 100 different aircraft models, with approximately 100 images provided for each model. In the field of fine-grained visual classification research, the Aircraft dataset is widely considered a key benchmark, especially for performance evaluation in aircraft model recognition tasks.

[0146] To maintain consistency with the CUB dataset experimental setup, the Aircraft dataset was processed using the same class division ratio, i.e., 100 aircraft models were randomly divided into 50 training classes, 25 validation classes, and 25 test classes. Furthermore, similar to the CUB dataset processing, images in the Aircraft dataset were cropped to ensure that the input images were uniformly focused on the aircraft models within the bounding boxes provided by the dataset. Considering that benchmark methods do not always employ pre-training in similar experimental settings, we decided not to implement pre-training in any fine-grained experiments to maintain consistency in experimental conditions and fairness in comparison results.

[0147] The feature reconstruction network was experimented on two widely used backbones: 4-layer Conv-4 and ResNet-12. Conv-4 consists of four consecutive 64-channel convolutional blocks, each downsampled by a factor of 2. Therefore, the output feature map shape for an input image of size 84×84 is 64×5×5. For ResNet12, the input image size is the same as Conv-4, and the output feature map shape is 640×5×5.

[0148] During training, the CUB dataset was used with 20 classes. The standard test during training was 1000 five-shot trials. The experiment used the SGD optimizer with an initial learning rate of 1e-1, momentum set to 0.9, and weight decay of 5e-4. The learning rate was reduced tenfold every 400 epochs, and validation was performed every 20 epochs to select the best-performing model. In contrast, Conv-4 was used as the backbone for 800 epochs, and ResNet-12 for 1200 epochs. After training, the standard one-shot and five-shot accuracies were obtained across 10,000 trials.

[0149] When compared with current leading few-shot learning methods such as the State of the Field (FRN) model and ProtoNet, this invention demonstrates significant performance on both raw and cropped input images. As shown in Tables 1 and 2, the best results are highlighted in bold. The "-" symbol indicates that the results cannot be obtained from the paper or code. The CUB dataset using frame-cropped images as input is abbreviated as CUB-Crop, while the CUB dataset using raw images as input is abbreviated as CUB-Raw.

[0150] Table 1: CUB-Crop Classification Test Results

[0151]

[0152]

[0153] Table 2: CUB-Raw Classification Test Results

[0154]

[0155] The implementation of the aircraft dataset largely follows the CUB setup. However, experiments showed that training the feature reconstruction network using the Adam optimizer yielded better results. The initial learning rate was set to 1e-3, with no weight decay, and other hyperparameters remained the same as in CUB. The test results are shown in Table 3.

[0156] Table 3: Aircraft classification test results.

[0157]

[0158] In the General Few-Shot benchmark test, this invention uses two datasets, mini-ImageNet and tiered-ImageNet, to evaluate the performance of the feature reconstruction network in the field of few-shot image classification. Both datasets originate from the widely recognized ImageNet dataset, but each has its own characteristics and is suitable for different experimental needs.

[0159] The mini-ImageNet dataset is a subset of the ImageNet dataset, containing 100 categories with 600 images per category, all at a uniform size of 84x84 pixels. In this experiment, these 100 categories were further divided: 64 categories for training, 16 for validation, and 20 for testing. Due to its diverse and balanced category distribution, mini-ImageNet is widely used in small-sample image classification research.

[0160] Tiered-ImageNet is also a larger subset of ImageNet, containing 34 superclasses and a total of 608 classes, with varying numbers of images per class. To ensure significant domain differences between training and evaluation, 351 classes were used for training, 97 for validation, and 160 for testing. Due to its large number and wide distribution of classes, tiered-ImageNet is suitable for testing model performance in more challenging multi-class few-shot learning scenarios. Most research methods use 84×84 pixel images. However, some models have been implemented at 224×224 pixel resolution. To ensure fairness in the experiments, this invention conducted experiments at both of these different resolution settings.

[0161] Recent research shows that pre-training on all training data and labels, followed by scenario-specific fine-tuning, can significantly improve model performance compared to training directly from scratch. When experimenting on the mini-ImageNet and tiered-ImageNet datasets, this invention pre-trains the model on the entire training set. This pre-training involves introducing a learnable matrix M for each class c into the feature reconstruction network beforehand. c As S c The agent.

[0162] During pre-training, the mini-ImageNet dataset was trained with a batch size of 128 for 350 epochs. The learning rate was reduced by a factor of 10 at the 200th and 300th epochs, and validation was performed every 25 epochs. Other testing methods remained consistent with CUB. In the fine-tuning phase, the same optimizer was used for 150 epochs, with validation performed every 5 epochs. The initial learning rate was 0.1, which was reduced by a factor of 10 at the 160th and 250th epochs.

[0163] Regardless of whether experiments were conducted on the 84×84 pixel tiered-ImageNet dataset or the 224×224 pixel dataset, the pre-training phase remained the same: a batch size of 128, 90 epochs of training, a tenfold reduction in the learning rate at the 30th and 60th epochs, and validation every 5 epochs. During fine-tuning, the total number of epochs on the 84×84 pixel dataset was 60, with the learning rate reduced by a tenfold at the 30th and 50th epochs, while the total number of epochs on the 224×224 pixel dataset was 70, with the learning rate reduced by a tenfold at the 40th and 60th epochs. The rest remained consistent with the mini-ImageNet dataset. The test results for the mini-ImageNet and tiered-ImageNet datasets are shown in Table 4, where ● indicates that the image resolution used in the tiered-ImageNet test was 224×224.

[0164] Table 4: Classification test results for mini-ImageNet and tiered-ImageNet.

[0165]

[0166]

[0167] In summary, based on the experimental data in Tables 1-4 above, it can be observed that the feature reconstruction network exhibits excellent performance in both fine-grained and general small-sample testing scenarios. In the comparative analysis, we found that in the 1-shot classification tasks of the CUB and Aircraft datasets, our invention using ResNet-12 demonstrates a significant advantage over FRN, with an accuracy improvement of 2.38% for CUB-RAW and 4.02% for Aircraft. Our invention also achieves significant progress in the 1-shot and 5-shot classification tasks of the mini-ImageNet and tiered-ImageNet datasets. Taking mini-ImageNet as an example, compared to OSLO, the accuracy of 1-shot classification is improved by 1.81%, and the accuracy of 5-shot classification is improved by 1.51%. On the more challenging tiered-ImageNet dataset, our method achieves accuracies of 77.89% and 87.72% in 1-shot and 5-shot learning, respectively, representing improvements of 5.32% and 0.83% in accuracy compared to FRN.

[0168] This invention evaluates a cross-domain setting, a crucial task for validating the adaptability and robustness of feature reconstruction networks when handling diverse data features and distributions. In this setting, the network is first trained on the base classes of mini-ImageNet and then evaluated on the test classes of CUB-Raw. Furthermore, the feature reconstruction network trained on tiered-ImageNet is applied to the CUB-Raw dataset for cross-domain experiments, and these results are compared with the cross-domain test results on mini-ImageNet. The results in Table 5 clearly demonstrate the effectiveness of the feature reconstruction network and its superior ability to handle cross-domain learning tasks.

[0169] Table 5: CUB classification results in cross-domain testing.

[0170]

[0171] The test results clearly demonstrate that the feature reconstruction network trained on the tiered-ImageNet dataset outperforms the feature reconstruction network trained on mini-ImageNet in cross-domain tests. This trend is evident in both 1-shot and 5-shot learning settings, where the performance conversion from tiered-ImageNet to CUB shows a 10% improvement in accuracy compared to the conversion from mini-ImageNet to CUB. This indicates that training on datasets with a wider range of categories and larger sample sizes can significantly improve the feature reconstruction network's generalization ability to novel and different datasets, enhancing its adaptability and flexibility when facing unknown data.

[0172] To verify the effectiveness of the proposed prototype completion and weighted smoothing focus loss function in the feature reconstruction network, several ablation experiments were conducted using the feature reconstruction network as a baseline model. To ensure fairness, ResNet-12 was selected as the backbone network for all experiments, and the same parameter settings were maintained across different module combinations. The results of each combination were recorded, such as... Figure 4 and Figure 5 As shown in the figure, experimental results demonstrate that combining prototype completion and the weighted smoothing focus loss function significantly reduces classification error, thereby enabling the feature reconstruction network to achieve optimal classification performance. This finding highlights the crucial role of prototype completion and the weighted smoothing focus loss function in improving the classification accuracy of feature reconstruction networks.

[0173] like Figure 6As shown, when training on a fine-grained benchmark dataset, the loss of the feature reconstruction network initially decreases rapidly, indicating that the network learns effectively from the data in the early stages. As training progresses, the loss decreases stepwise as the learning rate decays according to a predetermined strategy, reflecting the impact of learning rate adjustments on the training dynamics of the feature reconstruction network. After each decrease in the learning rate, the loss curve decreases smoothly or flattens, indicating that the feature reconstruction network is gradually approaching convergence. Adjusting the learning rate and weight decay positively contributes to the training of the feature reconstruction network.

[0174] like Figure 7 As shown, on the general benchmark dataset, the feature reconstruction network is less affected by the predetermined learning rate strategy decay, and the loss curve decreases smoothly to a flattening level.

[0175] Example 6

[0176] Corresponding to the methods mentioned above, such as Figure 8 As shown, this embodiment of the invention provides a few-shot learning system based on prototype completion and dynamic loss adjustment, including:

[0177] The new prototype generation module integrates information from the original support set and the original query set, generates a new prototype through tag propagation, and adds the new prototype to the original support set to form a new support set.

[0178] The feature map reconstruction module is used to reconstruct the query set based on the new support set using the feature map reconstruction network.

[0179] The training module is used to calculate the loss between the original query set and the reconstructed query set based on the constructed dynamic loss function, and to train the feature reconstruction network by minimizing the loss.

[0180] The classification module is used to input the dataset of images to be classified into a trained feature reconstruction network to obtain the image classification results.

[0181] It should be noted that the few-shot learning system based on prototype completion and dynamic loss adjustment provided in this embodiment of the invention is to implement the aforementioned few-shot learning method based on prototype completion and dynamic loss adjustment. Its specific functions can be referred to in the above-mentioned method embodiments, and will not be repeated here.

[0182] Example 7

[0183] Based on the above embodiments, this invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. The processor executes the computer program to implement a few-shot learning method based on prototype completion and dynamic loss adjustment as described in the above embodiments.

[0184] Preferably, the present invention also provides a computer-readable storage medium, the storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute a few-shot learning method based on prototype completion and dynamic loss adjustment as described in the above embodiments.

[0185] In summary, this invention expands the supporting samples of the prototype by using nearest neighbor samples between the support samples and the query samples. It updates the prototype using a weighted summation based on similarity, treating the new prototype as a support sample for data augmentation. Simultaneously, it applies a dynamic adjustment strategy for the loss function to focus the feature reconstruction network on samples that are difficult to classify, significantly improving the accuracy of small-sample classification tasks. By meticulously reducing intra-class and cross-class bias, it achieves accurate reconstruction and adjustment of the prototype, resulting in leading performance on multiple standard small-sample learning datasets. This invention introduces a prototype completion mechanism to capture key intra-class features, effectively generating more representative and accurate class prototypes, enhancing the understanding and representation of support set data. By utilizing the prototype-completed features in the feature reconstruction network for data augmentation, this invention effectively combines prototype completion and feature reconstruction techniques, enriching the training dataset and significantly improving adaptability and generalization ability to unknown categories through optimized feature representation. This invention employs a dynamic loss function adjustment strategy, which pays more attention to samples that are difficult to classify. This ensures that the feature reconstruction network can adaptively adjust during the learning process, and achieves personalized loss weight adjustments for samples of different difficulties. Thus, it improves classification accuracy while maintaining the robustness of the feature reconstruction network.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A few-shot learning method based on prototype completion and dynamic loss adjustment, characterized in that, Includes the following steps: Step 1: Integrate the information from the original support set and the original query set, generate a new prototype through label propagation, and add the new prototype to the original support set to form a new support set; Step 2: Reconstruct the network using feature maps to obtain the reconstructed query set based on the new support set; Step 3: Calculate the loss of the original query set and the reconstructed query set based on the constructed dynamic loss function, and train the feature map reconstruction network by minimizing the loss; Step 4: Input the dataset of images to be classified into the trained feature map reconstruction network to obtain the image classification results; Step three, the process of constructing the dynamic loss function, specifically includes: A weight adjustment matrix is ​​initialized based on one-hot encoding, and the output of the feature map reconstruction network is processed by softmax to obtain a probability matrix. The predicted probability of each sample to the true label is calculated based on the probability matrix, and all samples are traversed to update the weight adjustment matrix. If the predicted probability of the true label of a sample is lower than the set threshold, the weight of the category corresponding to the true label of the sample is increased, and the weight of other categories is decreased to obtain the new weight of the sample. Calculate the loss for each sample, and then use the weight adjustment matrix to summarize the weighted losses of all samples to obtain the total loss. The probability of the true label is expressed by the following formula: Where C is the total number of categories, It is the output of the feature map reconstruction network for the i-th sample in the j-th category. Let be the sample probability matrix of the i-th sample in the j-th class. Let be the predicted probability of the true label of the i-th sample; The weights of the samples are expressed by the following formula: in, Let be the weight corresponding to the i-th sample, C be the total number of categories, and eps be a very small constant. The loss for each sample can be expressed using the following formula: in, To control the weight parameters of positive and negative samples, Let C be the loss for the i-th sample, and C be the total number of classes. Let be the predicted probability of the true label of the i-th sample. Let be the sample probability matrix of the i-th sample in the j-th category.

2. The few-shot learning method based on prototype completion and dynamic loss adjustment according to claim 1, characterized in that, Step one specifically includes: Calculate the average feature offset between the original support set and the original query set, and correct the features of the original query set based on the average feature offset; Calculate the similarity between the original support set features and the corrected query set features and the prototype, and convert the similarity between the two into a probability distribution; The probability distributions of the original support set and the corrected query set are reweighted to obtain the reweighted probability distributions of the support set and the query set. The probability distributions of the reweighted support set and query set are applied to their respective features to update the prototype. The updated prototype is then added to the support set as a new sample to obtain a new support set.

3. The few-shot learning method based on prototype completion and dynamic loss adjustment according to claim 2, characterized in that, Calculate the corrected query set features using the following formula: in, Let i be the feature matrix of the i-th query sample. Let i be the feature matrix of the i-th supporting sample. The number of categories in the original support set. The number of categories in the original query set. and These are the average values ​​of the features in the original support set and the features in the original query set, respectively. These are the features of the corrected query set. Features of the original query set This is the difference between the average values ​​of the features in the original support set and the features in the original query set. The probability distributions of the original support set and the corrected query set are calculated using the following formula: in, and These represent the probability distributions of the original support set and the corrected query set relative to the prototype, respectively. cos is the cosine similarity function, and exp is the exponential function. Features of the original support set; Calculate the probability distribution of the reweighted support set and query set using the following formula: Where N is the normalized vector, and These are the sample sizes of the original support set and the corrected query set, respectively. and , respectively, are the labels of the i-th sample in the original support set and the corrected query set, where j is the number of categories. and These are the probability distributions of the original support set, the corrected query set, and the prototype, respectively. It is a one-hot encoding. This represents the probability distribution of the reweighted query set. Let be the probability distribution of the reweighted support set. Labels for the original support set or the corrected query set. For the tags of the original support set, Labels for the corrected query set; The new support set is calculated using the following formula: Where P is the updated prototype and S is the original support set. For the new support set.

4. A few-shot learning system based on prototype completion and dynamic loss adjustment, characterized in that, include: The new prototype generation module is used to integrate information from the original support set and the original query set, generate a new prototype through tag propagation, and add the new prototype to the original support set to form a new support set; The feature map reconstruction module is used to obtain a reconstructed query set based on the new support set using the feature map reconstruction network; The training module is used to calculate the loss between the original query set and the reconstructed query set based on the constructed dynamic loss function, and to train the feature map reconstruction network by minimizing the loss; The classification module is used to input the dataset of images to be classified into the trained feature map reconstruction network to obtain the image classification results; The dynamic loss function construction process in the training module specifically includes: A weight adjustment matrix is ​​initialized based on one-hot encoding, and the output of the feature map reconstruction network is processed by softmax to obtain a probability matrix. The predicted probability of each sample to the true label is calculated based on the probability matrix, and all samples are traversed to update the weight adjustment matrix. If the predicted probability of the true label of a sample is lower than the set threshold, the weight of the category corresponding to the true label of the sample is increased, and the weight of other categories is decreased to obtain the new weight of the sample. Calculate the loss for each sample, and then use the weight adjustment matrix to summarize the weighted losses of all samples to obtain the total loss. The probability of the true label is expressed by the following formula: Where C is the total number of categories, It is the output of the feature map reconstruction network for the i-th sample in the j-th category. Let be the sample probability matrix of the i-th sample in the j-th class. Let be the predicted probability of the true label of the i-th sample; The weights of the samples are expressed by the following formula: in, Let be the weight corresponding to the i-th sample, C be the total number of categories, and eps be a very small constant. The loss for each sample can be expressed using the following formula: in, To control the weight parameters of positive and negative samples, Let C be the loss for the i-th sample, and C be the total number of classes. Let be the predicted probability of the true label of the i-th sample. Let be the sample probability matrix of the i-th sample in the j-th category.

5. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a few-shot learning method based on prototype completion and dynamic loss adjustment as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform a few-shot learning method based on prototype completion and dynamic loss adjustment as described in any one of claims 1 to 3.

Images