Small sample learning method and system based on space self-adaption

Abstract

The invention discloses a small sample learning method and system based on space self-adaption. According to the method, global features are extracted by pre-training a backbone network, and global relation embedding between query samples and support categories is learned in a category-sensitive context, so that a global similarity score is obtained. And if the score is lower than the reliability threshold, recalculating the local similarity score by using the extended backbone network to obtain a more accurate classification result. By utilizing the method, the classification accuracy can be remarkably improved in the field of small sample learning, and the method has important application value.

Description

Technical Field

[0001] The present invention relates to a small sample learning method based on space adaptation, and also relates to a corresponding small sample learning system, belonging to the technical field of machine learning. Background Art

[0002] Few-shot learning is a machine learning problem characterized by the presence of only a limited amount of supervised information in the training dataset. The goal of this approach is to train a neural network model (referred to as the model) that can easily adapt to new categories with a small number of training samples, even if the model has strong generalization performance. In the field of deep learning, especially in computer vision and natural language processing, the study of few-shot learning has become a mainstream trend, because the acquisition of large-scale labeled training data is impractical for some application scenarios.

[0003] Existing small sample learning methods usually pre-train the feature extraction network on a large number of base class samples, which not only makes the training process more stable, but also speeds up the training and significantly improves the small sample classification performance. However, the pre-trained network usually uses global features and multiple classifiers for classification, which may ignore the discriminability of the local representation of the sample, making it difficult for these methods to distinguish similar samples. In addition, the fixed pre-trained backbone network is limited by the global cross entropy regularization and may extract invalid local features, which is particularly evident on new categories. Therefore, effectively extracting and utilizing local features is one of the keys to improving the small sample classification performance. In addition to the problem of feature extraction, the difference in distribution between the base class and the new class is also a challenge. The pre-trained feature extraction network needs to be fine-tuned on the new class to alleviate the overfitting caused by fine-tuning on the new class with a sparse number of samples, which is also one of the key issues to improve the small sample classification performance.

[0004] At the 2020 IJCAI conference, Yuqing Ma et al. published the paper "Few-shot Visual Learning with Contextual Memory and Fine-grained Calibration", which proposed a method of using the IPN network (Inverted Pyramid Network) of CCMNet (Class-sensitive Contextual Memory Network) for small-shot visual learning. The advantage of CCMNet is that it processes global features, but it is insufficient in capturing local features. To solve this problem, the IPN framework uses CCMNet to process global features in the global stage, followed by a local stage of fine-grained calibration to achieve more comprehensive feature capture and more accurate category prediction. However, the IPN framework uses CCM Net to capture features in both the global and local stages, which may affect the fine-grained calibration effect in the local stage. Therefore, combining global and local features is the key to improving the generalization ability and recognition accuracy of the model. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a small sample learning method based on space adaptation.

[0006] Another technical problem to be solved by the present invention is to provide a small sample learning system based on space adaptation.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a small sample learning method based on spatial adaptation is provided, comprising the following steps:

[0009] S1: Global feature extraction based on pre-trained backbone network;

[0010] S2: Based on the contextual information in a specific category, the global relation embedding between the query sample and the support category is learned in a class-sensitive context to obtain the global similarity score between the query sample and the support category;

[0011] S3: According to the global similarity score obtained in step S2, if the score is greater than or equal to the reliability threshold, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold, step S4 is entered to obtain the calibration classification result of the local stage;

[0012] S4: Send the information of the most similar multiple categories of the given query example to the extended backbone network, use the local features of the query sample to recalculate the local similarity score between the query sample and the support class, and compare it with the local threshold to obtain the classification result of the query sample.

[0013] Among them, the last layer of the pre-trained backbone network is a fully connected layer; the last layer of the extended backbone network is a spatial adaptation layer.

[0014] Preferably, the spatial adaptation layer is a convolutional layer, and the parameters will be dynamically adjusted according to the input features during the training process to adapt to distinguishing local fine-grained details.

[0015] Preferably, the spatial adaptation layer takes the one-dimensional feature vector output by the fully connected layer as input through a convolution operation to obtain a two-dimensional feature map.

[0016] Preferably, for sample x, the feature P extracted by the extended backbone network satisfies: in, is the selected ReLU activation function, Indicates an extended backbone network.

[0017] Preferably, the loss function is:

[0018]

[0019] Represents the local similarity score of the queried class.

[0020] Preferably, after step S4, the following steps are also included:

[0021] S5: A task-oriented fine-tuning strategy is adopted. For each task, the supporting features are averaged using the pre-trained backbone network architecture, the normalized features are used to calculate the prototype representation of each class, and these prototype representations are used to initialize the task-specific classifier;

[0022] S6: After task-oriented tuning, the relationship between the query and the class is calculated to distinguish its category.

[0023] Preferably, a task-oriented tuning strategy is adopted during the inference process, directly adjusting the parameters of the batch normalization layer in the pre-trained backbone network for each task, and using standard cross-entropy regularization to update the affine parameters of the batch normalization layer.

[0024] Preferably, the overall training function

[0025] Among them, l localrepresents the spatial adaptive loss function, l represents the task-oriented tuning strategy loss function,

[0026]

[0027] Among them, T represents the number of query samples; N represents the number of supported categories; K represents the number of supported samples, and the global similarity score Ψ represents the similarity measurement function, Indicates that the hidden states in both directions Concatenated together as the final global relation embedding.

[0028] According to a second aspect of an embodiment of the present invention, a small sample learning system based on spatial adaptation is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the above-mentioned small sample learning method based on spatial adaptation.

[0029] Compared with the prior art, the present invention achieves a more accurate similarity assessment between query samples and supported categories by transferring previously learned global and discriminative local feature knowledge to the current task, using a spatial adaptive layer to adaptively distinguish local fine-grained details, and maintaining the discriminative global representation capability of the pre-trained backbone network. This two-stage coarse-to-fine learning approach enables the model to effectively identify objects from global and local features with only a small number of samples. At the same time, the task-oriented tuning strategy helps the pre-trained backbone network adapt to specific new tasks with sparse data, further improving the performance of small sample learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 An overall flow chart of a small sample learning method based on spatial adaptation provided by an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of the BiGRU unit of CCMNet in an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of a backbone network in which the full support layer is replaced by a spatial adaptive layer in an embodiment of the present invention;

[0033] Figure 4 This is a comparison diagram of experimental results of a small sample learning method based on spatial adaptation in an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of the structure of a small sample learning system based on spatial adaptation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] The technical concept in the embodiment of the present invention is: an inverted pyramid network (Inverted Pyramid Network) is used for object recognition, and the network includes two stages: global and local. In the global stage, the network focuses on capturing the global features of the image; in the local stage, by introducing a spatial adaptation layer (SA layer), the network can highlight the local features of the sample and perform local feature calibration on the classification results obtained in the group stage to adaptively distinguish local fine-grained details. Such a design helps to maintain the discriminative global representation ability of the pre-trained backbone network. In addition, the present invention also proposes a task-oriented tuning strategy that can explicitly transfer knowledge from the base class to the new class based on the supporting samples of each task, thereby optimizing model performance. Figure 1 This process is shown, where the global stage is the initial stage of the network processing the input data, and the local stage is the subsequent stage of the network processing the input data, focusing on capturing the local features of the image.

[0037] First embodiment

[0038] like Figure 1 As shown, the first embodiment of the present invention provides a small sample learning method based on space adaptation, which at least includes the following steps.

[0039] S1: Perform global feature extraction based on the pre-trained backbone network (first backbone network).

[0040] Using pre-trained backbone network Extract global feature representations of query samples and support samples. It converts input data (such as images) into a low-dimensional, dense feature representation, thereby extracting useful information from high-dimensional data and compressing it into a more compact form for subsequent processing and analysis. The pre-trained backbone network is a convolutional neural network that has been pre-trained on a large dataset (such as ImageNet), which contains multiple convolutional layers, activation layers, pooling layers, and finally several fully connected layers (FC layers).

[0041] Among them, the fully connected layer is used to map the learned features to the final output space, such as the category label. The fully connected layer is used to integrate the features extracted by the previous convolutional layer or pooling layer for the final classification task. Each neuron in the fully connected layer is connected to all neurons in the previous layer, which means that it takes global information into account when integrating features. In addition, the fully connected layer has strict requirements on the size of the input feature map and cannot accept inputs of arbitrary size.

[0042] S2: Based on the contextual information in a specific category, the global relation embedding between the query sample and the support category is learned in a class-sensitive context to obtain the global similarity score between the query sample and the support category.

[0043] In one embodiment of the present invention, CCMNet uses multiple bidirectional gated recurrent units (BiGRU units for short) to gradually process query samples. and each supported category It initializes the hidden state as the query feature at each time step and inputs the support samples of each support category into the network to capture the relationship between the support samples and the query samples. Among them, CCMNet (class-specific contextual model with a memory mechanism) is a class-sensitive contextual memory network based on recurrent neural network (RNN).

[0044] Thus, the present invention realizes the use of specific categories and learn query samples in a class-sensitive context and support categories The embedding of the global relationship between them.

[0045] like Figure 2 As shown, CCMNet uses BiGRU units in its structure to process context information. In one embodiment of the present invention, CCMNet learns the temporal information and contextual relationships in the support sample sequence through BiGRU units. Specifically, the BiGRU unit processes a support sample at each time step. These support samples are samples of known categories, which are crucial to helping the model learn how to recognize new and unseen query samples.

[0046] CCMNet uses query features to initialize the hidden state of the BiGRU unit and inputs support samples of a specific class into the network at each time step. During processing, the BiGRU unit updates its hidden state, which contains information about all support samples processed so far. The BiGRU unit controls the flow of information through its gating mechanism and effectively processes sequence data. This mechanism enables the BiGRU unit to capture the contextual information of the support samples, and because the BiGRU unit processes the support samples in both forward and reverse order, it can learn the dependencies of the sequence data from two directions and provide a more comprehensive relationship representation for each time step. Such bidirectional processing enables the BiGRU unit to capture contextual information in different directions, enhancing the model's ability to understand sequence data.

[0047] CCMNet processes information by adopting a bidirectional mechanism of BiGRU units, which involves connecting the output hidden states of two opposite directions together. Specifically, the BiGRU unit processes the support samples not only in forward order, but also in reverse order to capture contextual information in different directions. In this way, the BiGRU unit is able to learn the dependencies of sequence data from two directions and combine this information to form a comprehensive relational embedding that includes both forward and reverse contextual information. This relational embedding is a mathematical representation of the relationship between the query sample and the support sample, which is used to capture the similarities or differences between the two.

[0048] In practice, CCMNet traverses all support samples belonging to each class and calculates the relationship embedding between the query sample and each support sample. These relationship embeddings are then used to evaluate the similarity between the query sample and each support class to obtain a similarity score. By comparing the similarities of all queries to classes for each support class, CCMNet is able to achieve relatively accurate classification. This approach enables CCMNet to effectively identify and classify new query samples in small sample learning scenarios, even with only a small number of support samples.

[0049] Specifically, the given query sample Input into the trained CCMNet to generate global relation embedding, and then calculate the global similarity score ( Ψ represents the similarity measure function) is used to support query pairs. represents the similarity score between the query sample q and the kth support sample in the nth class.

[0050] Among them, the relationship embedding is learned by comparing the updates of hidden states. The BiGRU unit adopts a bidirectional mechanism to update the hidden states in both directions. Concatenate them together as the final global relation embedding:

[0051]

[0052] The calculation process of global query to class similarity can be expressed as:

[0053]

[0054] Finally, the following task-oriented tuning strategy loss function is used to train the model to distinguish between different categories:

[0055]

[0056] S3: Based on the global similarity score obtained in step S2 If the score is greater than or equal to the reliability threshold τ0, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold τ0, step S4 is entered to obtain the calibration classification result of the local stage.

[0057] In order to further perform fine-grained calibration, the prediction reliability τ is calculated in the embodiment of the present invention. Specifically, assuming and If the two categories with the highest class similarity are queried, the prediction reliability τ of the task is calculated as follows:

[0058]

[0059] Compare the predicted reliability τ with the predetermined reliability threshold τ0. If τ ≥ τ0, directly take As the final predicted category of the query sample. Otherwise, the model is not very sure about the category of the query sample, and it is necessary to From and Category (i.e. the category that the model predicts in the global stage is most similar to the query sample and ) is sent to the extended backbone network in the local stage. In the local stage, after fine-grained calibration, a more accurate query class similarity is further obtained for the final prediction.

[0060] The reliability calculation formula is not affected by the task tuning strategy and is only related to the SA layer.

[0061] S4: The information of the given query example from the most similar multiple categories is sent to the extended backbone network, and the local similarity score between the query sample and the support class is recalculated using the local features of the query sample, and compared with the local threshold to obtain the classification result of the query sample.

[0062] In the present invention, the fully connected layer (FC layer) of the pre-trained backbone network is replaced by a spatial adaptive layer (SA layer), thereby forming an extended backbone network, i.e., a second backbone network. The SA layer is a convolutional layer, and its parameters are dynamically adjusted according to the input features during the training process to adapt to and distinguish local fine-grained details. This design enables the model to capture the global features of the image while retaining the front part of the pre-trained backbone network (usually the convolutional layer and the pooling layer), thereby maintaining the discriminative global representation ability of the pre-trained backbone network. At the same time, the dynamic adjustment capability of the SA layer enables the model to pay more attention to local areas in the image, which may contain key information to distinguish categories, and help to calibrate uncertain predictions in the global stage at the local stage, thereby improving the accuracy of classification.

[0063] Spatially adaptive layers (SA layers) can take many forms, including adaptive pooling layers, convolutional layers with attention mechanisms, or other types of deformable convolutional layers. The key feature of the SA layer is that it can adapt to the size of the input feature map. For example, the Global Average Pooling layer is a type of SA layer that performs an average pooling operation on the output (feature map) of the last convolutional layer to reduce the spatial dimensions (width W and height H) of each channel to a single value, thereby obtaining a fixed-length feature vector. This operation not only reduces the number of parameters of the model, but also enables the model to handle inputs of different sizes more flexibly, enhancing the model's ability to capture local features.

[0064] Unlike the fully connected layer in the pre-trained backbone network, which directly obtains a one-dimensional feature vector, the extended backbone network The SA layer in the pre-trained backbone network is transformed into The one-dimensional feature vector output by the last layer (fully connected layer) is used as input to obtain a two-dimensional feature map. Due to the deep number of layers, the feature map output by the SA layer mainly contains the local features of the target. Moreover, the convolution layer only needs to consider the size of the convolution kernel and does not need to pay attention to the dimension of the input features. Therefore, using a spatial adaptive layer to replace the fully connected layer can effectively capture local features, improve the adaptability of the local stage to changes in input size, reduce the number of parameters, and also reduce the risk of overfitting.

[0065] Since the present invention adopts a pre-trained backbone network in the global stage and an extended backbone network in the local stage, the use of CCMNet in the global stage is conducive to capturing global features, and the SA layer is used in the local stage to improve the ability to effectively capture local features.

[0066] In one embodiment of the present invention, in order to classify similar categories (e.g. Figure 1 The categories shown in and ) to distinguish, given a sample v (which is difficult to be distinguished by the global stage CCMNet in the pre-trained backbone network), by extending the backbone network Extract feature O: in, The ReLU activation function is selected because the spatial adaptation layer uses the ReLU activation function to introduce nonlinearity after adaptive feature processing, which is conducive to learning more complex feature representations while keeping the activation value of the network non-negative, which helps to improve the expression ability of the local stage and the stability of training.

[0067] like Figure 3 As shown, the backbone network will be expanded The output feature map of the last SA layer is regarded as a set of local patches O = {o1, ..., o M That is, for each local feature o of the query sample j , find its L nearest neighbors from all local patches of a class Then, the cosine similarity formula is used to calculate the similarity of all local patches of the nth class with their corresponding neighbors, and the similarity is added up to form the local similarity score of the queried class.

[0068]

[0069] Then, using regularization, the loss function used to supervise the training of the SA layer (i.e., the spatially adaptive loss function )as follows:

[0070]

[0071] When processing the data in the memory, for the same query sample, the output results generated in the current training process will be used to replace the output results obtained in the previous training process in the embodiment of the present invention. The purpose of this is to ensure that the data in the memory is always kept up to date and reflects the performance of the model in the latest training state.

[0072] Compared with the prior art, the embodiments of the present invention have the following technical features:

[0073] First, the CCMNet in this invention uses the classic GRU unit to process context information sequentially. In this process, CCMNet uses the query features to initialize the hidden state of the GRU unit and inputs the support samples of a specific category into the network at each time step. This design enables CCMNet to effectively process context information and update the hidden state at each time step to reflect the information of all support samples processed so far.

[0074] Secondly, by traversing all supporting samples belonging to each category, the present invention can calculate the similarity between the query sample and each supporting sample. This step is the key to evaluating the similarity between the query sample and each supporting category, and provides an important basis for subsequent classification decisions.

[0075] Thirdly, the present invention expands the backbone network and replaces the traditional fully connected layer with a spatial adaptive layer (SA layer). The SA layer is a convolutional layer whose parameters are dynamically adjusted according to the input features during the training process to adaptively distinguish local fine-grained details. This design allows the model to focus more on local areas in the image while retaining global features, which may contain key information to distinguish categories.

[0076] Finally, the present invention uses the local features of the query sample to recalculate the similarity between the query sample and the support class. This step helps to calibrate the prediction of the global stage at the local stage, thereby improving the accuracy of classification.

[0077] In summary, the present invention adjusts the SA layer to enable the inverted pyramid network (IPN) to obtain stronger representation capabilities from global and local perspectives. This design enables the model to effectively mine the relationship between discriminative queries and classes even when faced with a few-shot tasks containing similar categories, thereby improving the accuracy of classification.

[0078] Second embodiment

[0079] On the basis of the above-mentioned small sample learning method based on space adaptation, in order to further optimize this process, the second embodiment of the present invention adds a task-oriented tuning reasoning step S5 after step S4.

[0080] S5: A task-oriented fine-tuning strategy is adopted. For each task, the pre-trained backbone network is used, the supporting features are averaged, the normalized features are used to calculate the prototype representation of each class, and these prototype representations are used to initialize the task-specific classifier.

[0081] This additional step aims to fine-tune the model according to the needs of each specific task to improve the performance and accuracy of the model on specific tasks. Through such optimization, it can better adapt to new tasks with sparse data, thereby achieving better classification results in small sample learning scenarios.

[0082] Specifically, the present invention adopts a task-oriented tuning strategy during the inference process, which involves directly adjusting the parameters of the batch normalization layer in the pre-trained backbone network in each task. These parameters are calculated and stored as running estimates of the mean and variance during the training process, which are used to normalize the input data during inference. By adjusting these parameters, these statistics can be further optimized to make them more suitable for the data distribution of the current task. The parameters of the batch normalization layer are stored in the memory for use during inference.

[0083] Batch normalization is a method to mitigate internal covariate shift, which is introduced into deep learning models to accelerate model convergence and improve performance. By training the affine parameters of batch normalization and freezing other parameters, the performance of the model can be significantly improved. In few-shot learning tasks, fine-tuning these affine parameters based on the support samples of each task can quickly transform prior knowledge into the ability to recognize new unseen patterns. This fine-tuning strategy helps to match the discriminative representation extracted from the pre-trained backbone network with the data distribution of the new task, so as to better mine the relationship between query samples and categories and make accurate predictions.

[0084] For each task, the pre-trained backbone network is used to average the support features and normalize the features. Use conflict symbols here To represent the activated feature. The x in the following formula is the feature after the aforementioned activation function ReLU, which is scaled and shifted by the conventional batch normalization layer:

[0085]

[0086] Where γ and β are affine parameters, representing the scaling factor and offset respectively; E means taking the mean, $Var means finding the variance, ∈>0 and is a very small constant.

[0087] The affine parameters are updated using standard cross entropy regularization. Updating these parameters allows the model to adjust the normalized features.

[0088]

[0089] This can improve the performance of classification tasks.

[0090] S6: After task-oriented tuning, the relationship between the query and the class is calculated to distinguish its category.

[0091] After task-oriented tuning, combined with the spatial adaptive loss function And the aforementioned task-oriented tuning strategy loss function Get the overall training function Then, objects are classified using a small sample classification model trained with the overall training function.

[0092] Here, assuming and are the two categories with the highest similarity between the query and the class. The prediction reliability τ of the task is calculated as follows:

[0093]

[0094] Compare the predicted reliability τ with the predetermined reliability threshold τ0. If τ ≥ τ0, directly take As the final predicted category of the query sample. Otherwise, the given query example From and The categories are sent to the local stage through the SA layer. In this way, after fine-grained calibration, more accurate query class similarity is further obtained for final prediction.

[0095] The task-oriented tuning strategy loss function provided by the embodiment of the present invention has the following characteristics: first, for each task, the mean of the supporting features is calculated to represent the prototype of each class, and these prototypes are used to initialize the classifier of the task; second, the standard cross entropy regularization method is used to update the affine parameters of the batch normalization layer; then, after completing the task-oriented tuning, the given query sample is input into the trained CCMNet to generate the relation embedding and calculate the similarity score between the query sample and the supporting class; finally, in order to perform fine-grained calibration, the prediction reliability score is calculated. If the score is higher than the set threshold, the current classification result is considered valid; if the score is lower than the threshold, the sample is sent to the local stage for further calibration of the classification result. This process ensures the adaptability of the model to new tasks and the accuracy of classification.

[0096] In order to verify the effectiveness of the small sample learning method provided by the embodiment of the present invention, the inventors conducted extensive experiments on two benchmark datasets in the field of small sample learning, MiniImageNet and Tiered ImageNet. In the experiment, the method provided by the present invention was compared with some of the most advanced small sample learning methods currently available, including optimization-based methods, generation-based methods, and metric-based methods. The experimental results show that the method provided by the present invention significantly outperforms the existing most advanced small sample learning methods in terms of performance, as described below:

[0097] First, according to the number of new categories and corresponding samples, the small sample image classification is divided into two experimental settings: 5 categories (5-way) 1 sample (denoted as “1-shot” in the table below) and 5 samples (denoted as “5-shot” in the table below).

[0098] exist Figure 4In the comparison results shown, the method provided by the present invention (Ours) performs well on the Mini ImageNet dataset. Specifically, in the 1-shot setting, the method achieved an accuracy of 65.02%, and in the 5-shot setting, the accuracy reached 85.23%. Compared with previous methods, such as MGGN, our 5-shot setting accuracy is improved by about 1.94% (MGGN's accuracy is 83.29%). Similarly, on the Tiered ImageNet dataset, the method provided by the present invention achieved accuracies of 73.24% and 87.36% in the 1-shot and 5-shot settings, respectively. Compared with the GEL method, in the 1-shot setting, our accuracy is improved by 2.74% (GEL's accuracy is 70.50%).

[0099] These results show that, except in the 5-way 1-shot setting of Mini ImageNet, the method provided by the present invention has achieved the best in comparison with other state-of-the-art methods (SOTA). Compared with the results of the papers in the background technology, the method provided by the present invention also shows better performance. It should be noted that in the field of small sample classification, performance improvement is quite difficult. Therefore, the present invention achieved an improvement of 1.94% in the 5-way 5-shot setting of Mini ImageNet, and an improvement of 1.21% and 2.08% in the two experiments of Tiered ImageNet, respectively, showing obvious advantages.

[0100] In summary, the core of the present invention is to use a spatially adaptive inverted pyramid network (IPN) to comprehensively consider the global and discriminative local features of the pre-trained feature extractor network. First, through this network structure, the previously learned knowledge is transferred to the current task, so that the model can clearly identify and utilize global features as well as local features. Next, a spatial adaptive layer (SA layer) is introduced, which can adaptively distinguish local fine-grained details while helping to maintain the discriminative global representation ability of the pre-trained backbone network. Such a design enables the model to effectively identify objects from global and local features in a two-stage coarse-to-fine learning process, even with only a small number of samples, thereby obtaining a more accurate similarity assessment between the query sample and the support category.

[0101] On the other hand, the present invention also adopts a task-oriented tuning strategy, which fine-tunes the affine parameters in the pre-trained backbone network according to the support samples of each task to obtain better query-to-class relationships. This fine-tuning enables the representativeness of the model to adapt to specific new tasks with sparse data, thereby significantly improving the performance of small sample learning. In this way, the present invention can achieve more accurate classification and better performance in small sample learning scenarios.

[0102] On this basis, the overall training function is obtained by combining the spatial adaptive loss function and the task-oriented tuning strategy loss function A few-shot classification model trained with the population training function is able to classify the objects shown in the input image. This means that the model has learned how to recognize and distinguish between different object categories, and is able to apply this learned knowledge to new, unseen images to accurately classify the objects in the images.

[0103] Third embodiment

[0104] Based on the above-mentioned small sample learning method based on spatial adaptation, the third embodiment of the present invention provides a small sample learning system based on spatial adaptation. Figure 5 As shown, the small sample learning system includes one or more processors and a memory. The memory is coupled to the processor and is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the small sample learning method based on spatial adaptation as in the above embodiment.

[0105] Among them, the processor is used to control the overall operation of the small sample learning system to complete all or part of the steps of the above-mentioned small sample learning method based on spatial adaptation. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support the operation of the small sample learning system, and these data may include, for example, instructions for any application or method for operating on the small sample learning system, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc.

[0106] In an exemplary embodiment, the small sample learning system can be implemented by a computer chip or entity, or by a product with a certain function, for executing the above-mentioned small sample learning method based on spatial adaptation, and achieving the same technical effect as the above-mentioned method. A typical embodiment is a computer, a server. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] In another exemplary embodiment, the present invention further provides a computer-readable storage medium including program instructions, which, when executed by a processor, implements the steps of the spatially adaptive small sample learning method in any of the above embodiments. For example, the computer-readable storage medium may be the above-mentioned memory including program instructions, and the above-mentioned program instructions may be executed by a processor of a system to complete the above-mentioned spatially adaptive small sample learning method and achieve the same technical effect as the above-mentioned method.

[0108] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined, and the order of the steps can be changed, all within the protection scope of the present invention.

[0109] The above is a detailed description of the spatially adaptive small sample learning method and system provided by the present invention. For those skilled in the art, any obvious changes made to the present invention without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear the corresponding legal liability.

Claims

1. A small sample learning method based on spatial adaptation, characterized in that The following steps are involved: S1: Global feature extraction based on pre-trained backbone network; S2: Based on the contextual information in a specific category, the global relation embedding between the query sample and the support category is learned in a class-sensitive context to obtain the global similarity score between the query sample and the support category; S3: According to the global similarity score obtained in step S2, if the score is greater than or equal to the reliability threshold, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold, step S4 is entered to obtain the calibration classification result of the local stage; S4: Send the information of the most similar multiple categories of the given query example to the extended backbone network, use the local features of the query sample to recalculate the local similarity score between the query sample and the support class, and compare it with the local threshold to obtain the classification result of the query sample; Among them, the last layer of the pre-trained backbone network is a fully connected layer; the last layer of the extended backbone network is a spatial adaptation layer.

2. The spatially adaptive small sample learning method according to claim 1, characterized in that: The spatial adaptation layer is a convolutional layer, and its parameters will be dynamically adjusted according to the input features during the training process to adapt to distinguishing local fine-grained details.

3. The spatially adaptive small sample learning method according to claim 2, characterized in that: The spatial adaptation layer uses the one-dimensional feature vector output by the fully connected layer as input through a convolution operation to obtain a two-dimensional feature map.

4. The spatially adaptive small sample learning method according to claim 3, characterized in that For sample x, the extended backbone network extracts features P that satisfy: in, is the selected ReLU activation function, Indicates an extended backbone network.

5. The spatial adaptive small sample learning method according to claim 4, characterized in that the space The adaptive loss function is: Represents the local similarity score of the queried class.

6. The spatially adaptive small sample learning method according to any one of claims 1 to 5, characterized in that After step S4, the following steps are also included: S5: A task-oriented fine-tuning strategy is adopted. For each task, the supporting features are averaged using the pre-trained backbone network, the normalized features are used to calculate the prototype representation of each class, and these prototype representations are used to initialize the task-specific classifier.

7. The spatially adaptive small sample learning method according to claim 6, characterized in that The following steps are also included: S6: After task-oriented tuning, the relationship between the query and the class is calculated to distinguish its category.

8. The spatially adaptive small sample learning method according to claim 6, characterized in that: A task-oriented tuning strategy is adopted during inference, directly adjusting the parameters of the batch normalization layer in the pre-trained backbone network for each task, and using standard cross-entropy regularization to update the affine parameters of the batch normalization layer.

9. The spatially adaptive small sample learning method according to claim 6, characterized in that Overall training function l total =l+l local , Among them, l local represents the spatial adaptive loss function, l represents the task-oriented tuning strategy loss function, Among them, T represents the number of query samples; N represents the number of supported categories; K represents the number of supported samples, and the global similarity score Ψ represents the similarity measurement function, Indicates that the hidden states in both directions Concatenated together as the final global relation embedding.

10. A small sample learning system based on spatial adaptation, characterized in that It comprises a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the small sample learning method based on spatial adaptation as described in any one of claims 1 to 9.

Images