Small sample learning method and system based on spatial adaptation

Abstract

The application discloses a small sample learning method and system based on space self-adaption. The method extracts global features through pre-training of a main network, and learns global relationship embedding between a query sample and support categories in a class-sensitive context to obtain a global similarity score. If the score is lower than a reliability threshold, a local similarity score is recalculated by using an extended main network to obtain a more accurate classification result. By using the application, the classification accuracy can be significantly improved in the field of small sample learning, and the application has important application value.

Description

TECHNICAL FIELD

[0001] The present application relates to a small sample learning method based on spatial adaptation, and also relates to a corresponding small sample learning system, and belongs to the technical field of machine learning. BACKGROUND

[0002] Few-shot Learning is a machine learning problem characterized by containing only a limited amount of supervised information in the training dataset. The goal of this method is to train a neural network model (referred to as model for short) that can easily adapt to new classes through a small number of samples, even if the model has strong generalization performance. In the field of deep learning, especially in computer vision and natural language processing, the study of few-shot learning has become a mainstream trend because the acquisition of large-scale labeled training data is impractical for certain application scenarios.

[0003] Existing few-shot learning methods usually pre-train the feature extraction network on a large number of base class samples. This not only makes the training process more stable, but also speeds up the training and significantly improves the few-shot classification performance. However, the pre-trained network usually uses global features and multi-classifiers for classification, which may ignore the discriminative nature of local sample representations, making it difficult for these methods to distinguish similar samples. In addition, the fixed pre-trained backbone network is limited by global cross-entropy regularization, which may extract invalid local features, and this problem is particularly evident on new classes. Therefore, effectively extracting and utilizing local features is one of the key factors to improve the performance of few-shot classification. In addition to the problem of feature extraction, the distribution difference between base classes and new classes is also a challenge. The pre-trained feature extraction network needs to be fine-tuned on new classes to alleviate the overfitting problem when fine-tuning on new classes with sparse sample quantities, which is also one of the key problems to improve the performance of few-shot classification.

[0004] In the IJCAI conference in 2020, Yuqing Ma et al. published the paper "Few-shot Visual Learning with Contextual Memory and Fine-grained Calibration", which proposed a method of small sample visual learning using IPN network (Inverted Pyramid Network) of CCMNet (Class-sensitive Contextual Memory Network). The advantage of CCMNet is to process global features, but it is insufficient in capturing local features. In order to solve this problem, the IPN framework will connect the fine-grained calibration local stage after the global stage of using CCMNet to process global features, so as to realize more comprehensive feature capture and more accurate class prediction. However, in the IPN framework, CCMNet is used to capture features in both the global stage and the local stage, which may affect the fine-grained calibration effect of the local stage. Therefore, combining global and local features is the key to improving the generalization ability and recognition accuracy of the model. SUMMARY

[0005] The primary technical problem to be solved by the present application is to provide a small sample learning method based on spatial adaptation.

[0006] Another technical problem to be solved by the present application is to provide a small sample learning system based on spatial adaptation.

[0007] To achieve the above technical purpose, the technical scheme adopted by the present application is as follows:

[0008] According to the first aspect of the embodiment of the present application, a small sample learning method based on spatial adaptation is provided, comprising the following steps:

[0009] S1: global feature extraction based on a pre-trained backbone network;

[0010] S2: learning the global relationship embedding between the query sample and the support class in the class-sensitive context based on the context information in the specific class, to obtain the global similarity score between the query sample and the support class;

[0011] S3: according to the global similarity score obtained in step S2, if the score is greater than or equal to the reliability threshold, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold, step S4 is entered to obtain the calibration classification result of the local stage;

[0012] S4: feeding the information from the most similar multiple categories of the given query example into the extended backbone network, re-computing the local similarity scores between the query sample and the support classes using the local features of the query sample, and comparing with the local threshold to obtain the classification result of the query sample,

[0013] wherein the last layer of the pre-trained backbone network is a fully connected layer; and the last layer of the extended backbone network is a spatial adaptive layer.

[0014] Preferably, the spatial adaptive layer is a convolution layer, and the parameters will be dynamically adjusted according to the input features during the training process to adapt to the local fine-grained details.

[0015] Preferably, the spatial adaptive layer takes the one-dimensional feature vector output by the fully connected layer as input through convolution operation to obtain a two-dimensional feature map.

[0016] Preferably, for a sample x, the extended backbone network extracts features P satisfying: wherein, is a selected ReLU activation function, denotes the extended backbone network.

[0017] Preferably, the loss function is:

[0018]

[0019] denotes the local similarity score of the query to the class.

[0020] Preferably, after the step S4, the method further comprises the following steps:

[0021] S5: using a task-oriented tuning strategy, for each task, using the pre-trained backbone network architecture to average the support features, using the normalized features to calculate the prototype representation of each class, and using these prototype representations to initialize the task-specific classifier;

[0022] S6: after the task-oriented tuning, calculating the relationship between the query and the class to distinguish the class.

[0023] Preferably, in the inference process, the task-oriented tuning strategy is used to directly adjust the parameters of the batch normalization layer in the pre-trained backbone network for each task, and the standard cross-entropy regularization is used to update the affine parameters of the batch normalization layer.

[0024] Preferably, the overall training function is

[0025] wherein, l localdenotes a spatially adaptive loss function, l denotes a task-oriented tuning strategy loss function,

[0026]

[0027] where T denotes the number of query samples; N denotes the number of support classes; K denotes the number of support samples, the global similarity score Ψ denotes a similarity measure function, denotes concatenating the hidden states in both directions to form the final global relation embedding.

[0028] According to a second aspect of the embodiments of the present application, a small sample learning system based on spatial adaptation is provided, comprising a processor and a memory; wherein the memory is coupled to the processor, and is used to store a computer program, when the computer program is executed by the processor, the processor realizes the small sample learning method based on spatial adaptation.

[0029] Compared with the prior art, the present application transfers the previously learned global and discriminative local feature knowledge to the current task, uses the spatial adaptation layer to adaptively distinguish local fine-grained details, and maintains the discriminative global representation ability of the pre-trained backbone network, so as to realize more accurate similarity evaluation between query samples and support classes. This two-stage coarse-to-fine learning method enables the model to effectively identify objects from global and local features with only a small number of samples. At the same time, the task-oriented tuning strategy helps the pre-trained backbone network to adapt to specific new tasks with sparse data, further improving the performance of small sample learning. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The overall flowchart of the small sample learning method based on spatial adaptation provided by the embodiments of the present application is shown in the figure.

[0031] Figure 2 The structure diagram of the BiGRU unit of the CCMNet in the embodiments of the present application is shown in the figure.

[0032] Figure 3 The backbone network diagram in which the global support layer is replaced by the spatial adaptation layer in the embodiments of the present application is shown in the figure.

[0033] Figure 4 The experimental effect comparison diagram of the small sample learning method based on spatial adaptation in the embodiments of the present application is shown in the figure.

[0034] Figure 5 The structure diagram of the small sample learning system based on spatial adaptation provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0035] The technical content of the present application will be described in detail below with reference to the drawings and specific embodiments.

[0036] The technical concept in the embodiment of the present application is to use an inverted pyramid network for object recognition, which includes global and local two stages. In the global stage, the network focuses on capturing the global features of the image; while in the local stage, by introducing a spatial adaptive layer (SA layer), the network can highlight the local features of the sample and calibrate the classification results obtained in the group stage to adaptively distinguish local fine-grained details. Such design helps to maintain the discriminative global representation ability of the pre-trained backbone network. In addition, the present application also proposes a task-oriented optimization strategy, which can explicitly transfer knowledge from the base class to the new class according to the support samples of each task, thereby optimizing the model performance. Figure 1 This process is shown, where the global stage (Global Stage) is the initial stage of the network processing input data, and the local stage (Local Stage) is the subsequent stage of the network processing input data, focusing on capturing the local features of the image.

[0037] First embodiment

[0038] As Figure 1 shown, the first embodiment of the present application provides a small sample learning method based on spatial adaptation, which at least includes the following steps.

[0039] S1: Global feature extraction based on a pre-trained backbone network (first backbone network).

[0040] Global feature representation of query samples and support samples is extracted using a pre-trained backbone network . It converts input data (such as images) into a low-dimensional, dense feature representation, which extracts useful information from high-dimensional data and compresses it into a more compact form for subsequent processing and analysis. The pre-trained backbone network is a convolutional neural network that has been pre-trained on a large dataset (such as ImageNet), containing multiple convolutional layers, activation layers, pooling layers, and finally several fully connected layers (FC layers).

[0041] Among them, the fully connected layer is used to map the learned features to the final output space, such as class labels. The fully connected layer is used to integrate the features extracted by the previous convolutional or pooling layer to perform the final classification task. Each neuron of the fully connected layer has a connection with all the neurons of the previous layer, which means it considers global information when integrating features. In addition, the fully connected layer has strict requirements for the size of the input feature map and cannot accept any size of input.

[0042] S2: learning global relationship embedding between query samples and support classes in class-sensitive context based on context information in specific class, to obtain global similarity score between query samples and support classes.

[0043] In an embodiment of the present application, the CCMNet adopts multiple bidirectional gated recurrent units (BiGRU) to process query samples and support samples of each support class step by step. It initializes the hidden state as query features at each time step and inputs support samples of each support class into the network to capture the relationship between support samples and query samples. The CCMNet (class-specific contextual model with a memory mechanism) is a class-sensitive context memory network based on recurrent neural network (RNN).

[0044] Thus, the present application realizes learning global relationship embedding between query samples and support classes in class-sensitive context based on context information in specific class.

[0045] As shown in Figure 2 , the CCMNet adopts BiGRU units in its structure to process context information. In an embodiment of the present application, the CCMNet learns the timing information and context relationship in the support sample sequence through BiGRU units. Specifically, the BiGRU unit processes one support sample at each time step, and these support samples are samples of known classes, which are crucial for helping the model learn how to identify new and unseen query samples.

[0046] The CCMNet uses query features to initialize the hidden state of the BiGRU unit and inputs support samples of a specific class into the network at each time step. During processing, the BiGRU unit updates its hidden state, which contains information about all support samples processed so far. The BiGRU unit controls the flow of information through its gating mechanism, effectively processing sequence data. This mechanism enables the BiGRU unit to capture the context information in different directions and enhances the model's understanding of sequence data. ​​

[0047] CCMNet processes information by employing a bidirectional mechanism of BiGRU units, which involves concatenating the output hidden states of two opposite directions together. Specifically, BiGRU units not only process support samples in a forward order, but also in a reverse order to capture the context information in different directions. In this way, BiGRU units are able to learn the dependency of sequence data from two directions and integrate the information to form a comprehensive relationship embedding containing the forward and reverse context information. This relationship embedding is a mathematical representation of the relationship between the query sample and the support samples, which is used to capture the similarity or difference between them.

[0048] In actual operation, CCMNet traverses all support samples belonging to each class and calculates the relationship embedding between the query sample and each support sample. These relationship embeddings are then used to evaluate the similarity between the query sample and each support class, resulting in a similarity score. By comparing the query-to-class similarity of each support class, CCMNet is able to achieve relatively accurate classification. This approach enables CCMNet to effectively identify and classify new query samples even in small sample learning scenarios, where only a small number of support samples are available.

[0049] Specifically, a given query sample is input into the trained CCMNet to generate a global relationship embedding, and then a global similarity score ( Ψ represents a similarity measure function) is calculated for the support query pair. denotes the similarity score between the query sample q and the kth support sample in the nth class.

[0050] where the relationship embedding is learned by comparing the updates of hidden states, and the BiGRU unit employs a bidirectional mechanism to concatenate the hidden states in two directions as the final global relationship embedding:

[0051]

[0052] The calculation process of global query-to-class similarity can be represented as:

[0053]

[0054] Finally, the model is trained using the following task-oriented tuning strategy loss function to enable it to distinguish between different classes:

[0055]

[0056] S3: According to the global similarity score obtained in step S2 ​If the score is greater than or equal to the reliability threshold τ0, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold τ0, step S4 is entered to obtain the calibration classification result of the local stage.

[0057] To further perform fine-grained calibration, the prediction reliability τ is calculated in the embodiment of the application. Specifically, assuming and are the two categories with the highest query-to-category similarity, the prediction reliability τ of the task is calculated as follows:

[0058]

[0059] The prediction reliability τ is compared with a predetermined reliability threshold τ0. If τ ≥ τ0, the is directly taken as the final prediction category of the query sample. Otherwise, the model is not very sure about the category attribution of the query sample, and the given query example from and categories (i.e., the categories most similar to the query sample predicted by the model in the global stage and ) is sent to the extended backbone network in the local stage. In the local stage, after fine-grained calibration, more accurate query category similarity is further obtained to make the final prediction.

[0060] The reliability calculation formula is not affected by the task tuning strategy, but only related to the SA layer.

[0061] S4: The information of the given query example from the most similar multiple categories is sent to the extended backbone network, the local similarity scores between the query sample and the support categories are recalculated using the local features of the query sample, and the classification result of the query sample is obtained by comparing the local threshold.

[0062] In the application, the fully connected layer (FC layer) of the pre-trained backbone network is replaced by a spatial adaptive layer (SA layer), thereby forming an extended backbone network, i.e., a second backbone network. The SA layer is a convolutional layer, and its parameters are dynamically adjusted according to the input features during the training process to adapt to and distinguish local fine-grained details. This design enables the model to retain the global feature of the image based on the front part of the pre-trained backbone network (usually convolutional layers and pooling layers), while still being able to capture the global features of the image, thereby maintaining the discriminative global representation ability of the pre-trained backbone network. At the same time, the dynamic adjustment ability of the SA layer enables the model to pay more attention to local regions in the image, which may contain key information of the category, and helps to calibrate the uncertain prediction in the local stage, thereby improving the classification accuracy.

[0063] The spatial adaptive layer (SA layer) can be in various forms, including an adaptive pooling layer, a convolution layer with an attention mechanism, or other types of deformable convolution layers. The key feature of the SA layer is that it can adaptively adjust the size of the input feature map. For example, a global average pooling layer is a type of SA layer that reduces the spatial dimensions (width W and height H) of each channel to a single value by performing an average pooling operation on the output (feature map) of the last convolution layer, thereby obtaining a fixed-length feature vector. This operation not only reduces the number of parameters in the model, but also enables the model to more flexibly handle different sizes of input, enhancing the model's ability to capture local features.

[0064] Unlike the fully connected layer in the pre-trained backbone network, which directly obtains a one-dimensional feature vector, the SA layer in the extended backbone network obtains a two-dimensional feature map by performing a convolution operation on the one-dimensional feature vector output by the last layer (fully connected layer) of the pre-trained backbone network . Due to the deep layer structure, the feature map output by the SA layer mainly contains local features of the target. Moreover, the convolution layer only needs to consider the size of the convolution kernel and does not need to consider the dimensions of the input features. Therefore, replacing the fully connected layer with the spatial adaptive layer can effectively capture local features, improve the adaptability of the local stage to changes in input size, reduce the number of parameters, and also reduce the risk of overfitting.

[0065] Since the present application uses a pre-trained backbone network in the global stage and an extended backbone network in the local stage, the use of CCMNet in the global stage is beneficial for capturing global features, and the use of the SA layer in the local stage improves the ability to effectively capture local features.

[0066] In an embodiment of the present application, in order to distinguish similar classes (such as the classes shown in Figure 1 and ), a sample v (which is difficult to distinguish by the CCMNet in the global stage of the pre-trained backbone network) is given, and the feature O is extracted by the extended backbone network : wherein, is a selected ReLU activation function. Since the spatial adaptive layer uses the ReLU activation function to introduce nonlinearity after adaptive feature processing, this is beneficial for learning more complex feature representations while keeping the activation values of the network non-negative, which helps to improve the expression ability of the local stage and the stability of the training.

[0067] As shown in Figure 3 , the extended backbone network ​the output feature maps outputted by the last SA layer are regarded as a set of local patches O = {o1, …, o M}. That is, each local feature o j The L nearest neighbors of the query sample are found from all local patches of a certain class Then, the similarity between all local patches of the nth class and their corresponding neighbors is calculated by the cosine similarity formula, and the sum of them is the local similarity score of the query to the class

[0068]

[0069] Then, the regularization is used for the loss function (i.e., the spatial adaptive loss function ) used for the training of the SA layer as follows:

[0070]

[0071] In processing the data in the memory, the output results generated in the current training process are used to replace the output results obtained in the previous training process for the same query sample. The purpose of this is to ensure that the data in the memory always remains up-to-date, reflecting the performance of the model in the latest training state.

[0072] Compared with the prior art, the embodiments of the present application have the following technical features:

[0073] Firstly, the CCMNet in the present application uses the classic GRU unit to sequentially process the context information. In this process, the CCMNet uses the query feature to initialize the hidden state of the GRU unit, and inputs the support samples of a certain class into the network at each time step. This design enables the CCMNet to effectively process the context information and update the hidden state at each time step to reflect the information of all support samples processed so far.

[0074] Secondly, by traversing all support samples belonging to each class, the present application can calculate the similarity between the query sample and each support sample. This step is the key to evaluating the similarity between the query sample and each support class, and provides an important basis for subsequent classification decisions.

[0075] Thirdly, the present application extends the backbone network, replacing the traditional fully connected layer with a spatial adaptive layer (SA layer). The SA layer is a convolutional layer, and its parameters will be dynamically adjusted during the training process according to the input features to adaptively distinguish local fine-grained details. Such a design enables the model to pay more attention to local regions in the image, which may contain key information for classifying the class, while preserving the global features.

[0076] Finally, the application re-computes the similarity between the query sample and the support classes using the local features of the query sample. This step helps to calibrate the prediction of the global stage in the local stage, thereby improving the accuracy of classification.

[0077] In summary, the application adjusts the SA layer to enable the Inverted Pyramid Network (IPN) to obtain stronger representation ability from both global and local perspectives. This design enables the model to effectively mine the relationship between the query and the class even when facing tasks containing a few shots of similar categories, thereby improving the accuracy of classification.

[0078] Second embodiment

[0079] Based on the above-mentioned spatial adaptive small sample learning method, in order to further optimize this process, the second embodiment of the application adds a task-oriented optimization reasoning step S5 after step S4.

[0080] S5: A task-oriented optimization strategy is adopted, for each task, the pre-trained backbone network is used to average the support features, the normalized features are used to calculate the prototype representation of each class, and the prototype representation is used to initialize the task-specific classifier.

[0081] This new step aims to fine-tune the model according to the needs of each specific task to improve the performance and accuracy of the model on specific tasks. Through such optimization, it can better adapt to new tasks with sparse data, thereby achieving better classification results in the small sample learning scenario.

[0082] Specifically, the application adopts a task-oriented optimization strategy in the reasoning process, which involves directly adjusting the parameters of the Batch Normalization layer within the pre-trained backbone network in each task. These parameters are calculated and stored as running estimates of mean and variance during training, which are used to normalize input data during inference. By adjusting these parameters, these statistics can be further optimized to better adapt to the data distribution of the current task. The parameters of the batch normalization layer are saved in memory for use during inference.

[0083] Batch normalization is a method to alleviate internal covariate shift, which is introduced into deep learning models to accelerate the convergence of the model and improve performance. By training the affine parameters of batch normalization and freezing other parameters, the performance of the model can be significantly improved. In few-shot learning tasks, fine-tuning these affine parameters according to the support samples of each task can quickly convert the previous knowledge into the ability to recognize new patterns that have not been seen before. This fine-tuning strategy helps to match the discriminative representation extracted in the pre-trained backbone network with the data distribution of the new task, so as to better mine the relationship between the query sample and the class and make accurate predictions.

[0084] For each task, the pre-trained backbone network is used to average the support features, and the normalized features Here, the conflict symbol is used to represent the activation feature. In the following formula, x is the feature after the activation function ReLU, which is scaled and shifted by the conventional batch normalization layer:

[0085]

[0086] Where γ and β are affine parameters, representing the scaling factor and offset, respectively; E represents the mean, Var represents the variance, ∈ > 0 and is a very small constant.

[0087] The affine parameters are updated using standard cross-entropy regularization. Updating these parameters allows the model to adjust the normalized features

[0088]

[0089] This can improve the performance of the classification task.

[0090] S6: After task-oriented tuning, the relationship between the query and the class is calculated to distinguish the class.

[0091] After task-oriented tuning, the spatial adaptive loss function and the aforementioned task-oriented tuning strategy loss function The overall training function is obtained Then, the small sample classification model trained by the overall training function is used for object classification.

[0092] Here, it is assumed that and are the two classes with the highest similarity between the query and the class, and the prediction reliability τ of the task is calculated as follows:

[0093]

[0094] The predicted reliability τ is compared with a predetermined reliability threshold τ 0. If τ ≥ τ 0, the final prediction class is directly taken The final prediction class of the query sample. Otherwise, the given query sample from and The class is sent to the local stage by the SA layer. In this way, after fine-grained calibration, further more accurate query class similarity is obtained to make the final prediction.

[0095] The task-oriented tuning strategy loss function provided by the embodiment of the present application has the following characteristics: first, for each task, the mean of the support features is calculated to represent the prototype of each class, and these prototypes are used to initialize the classifier of the task; second, the standard cross-entropy regularization method is used to update the affine parameters of the batch normalization layer; then, after completing the task-oriented tuning, the given query sample is input into the trained CCMNet to generate relationship embedding and calculate the similarity score of the query sample and the support class; finally, in order to perform fine-grained calibration, the predicted reliability score is calculated, and if the score is higher than the set threshold, the current classification result is considered valid; if the score is lower than the threshold, the sample is sent to the local stage for further calibration of the classification result. This process ensures the adaptability of the model on new tasks and the accuracy of classification.

[0096] In order to verify the effectiveness of the small sample learning method provided by the embodiment of the present application, the inventors have carried out extensive experiments on two benchmark datasets in the field of small sample learning, MiniImageNet and Tiered ImageNet. In the experiment, the method provided by the present application is compared with some of the most advanced small sample learning methods, including optimization-based methods, generation-based methods and metric-based methods. The experimental results show that the method provided by the present application is significantly better than the existing most advanced small sample learning methods in performance, which is specifically explained as follows:

[0097] First, according to the number of new classes and corresponding samples, small sample image classification is divided into 5 categories (5-way) 1 sample (denoted as "1-shot" in the table below) and 5 samples (denoted as "5-shot" in the table below) two experimental settings.

[0098] In Figure 4In the shown comparison results, the method provided by the application (Ours) performs well on the Mini ImageNet dataset. Specifically, in the 1-shot setting, the method achieves an accuracy of 65.02%, and in the 5-shot setting, the accuracy reaches 85.23%. Compared with previous methods, for example, MGGN, our 5-shot setting accuracy is increased by about 1.94% (the accuracy of MGGN is 83.29%). Similarly, on the Tiered ImageNet dataset, the method provided by the application achieves an accuracy of 73.24% and 87.36% in the 1-shot and 5-shot settings, respectively. Compared with the GEL method, in the 1-shot setting, our accuracy is increased by 2.74% (the accuracy of GEL is 70.50%).

[0099] These results show that, except in the 5-way 1-shot setting of Mini ImageNet, the method provided by the application achieves the best in comparison with other state-of-the-art (SOTA) methods. Compared with the results of the papers in the background art, the method provided by the application also shows better performance. It should be noted that in the field of small sample classification, performance improvement is quite difficult. Therefore, the 1.94% improvement in the 5-way 5-shot setting of Mini ImageNet and the 1.21% and 2.08% improvements in the two experiments of Tiered ImageNet respectively all show obvious advantages.

[0100] In summary, the core of the application is to use a spatially adaptive inverse pyramid network (IPN) to comprehensively consider the global and discriminative local features of the pre-trained feature extractor network. First, through this network structure, the previously learned knowledge is transferred to the current task, so that the model can explicitly identify and utilize global features and local features. Then, the spatial adaptive layer (SA layer) is introduced, which can adaptively distinguish local fine-grained details while helping to maintain the discriminative global representation ability of the pre-trained backbone network. Such design enables the model to effectively identify objects from global and local features in a two-stage coarse-to-fine learning process, even with only a small number of samples, thereby obtaining more accurate similarity evaluation between query samples and support classes.

[0101] On the other hand, the present application also adopts a task-oriented tuning strategy, which adjusts the affine parameters in the pre-trained backbone network according to the support samples of each task to obtain a better query-to-class relationship through fine-tuning. This fine-tuning enables the representational capacity of the model to adapt to the specific new task with sparse data, thereby significantly improving the performance of small sample learning. Through this method, the present application can achieve more accurate classification and better performance in the small sample learning scenario.

[0102] On this basis, the overall training function is obtained by combining the spatial adaptive loss function and the task-oriented tuning strategy loss function The small sample classification model trained by the overall training function can classify the objects displayed in the input image. This means that the model has learned how to recognize and distinguish different object categories and can apply this learned knowledge to new, unseen images to accurately classify the objects in the images.

[0103] Third embodiment

[0104] On the basis of the above-mentioned spatial adaptive small sample learning method, the third embodiment of the present application provides a spatial adaptive small sample learning system. As shown in Figure 5 The small sample learning system includes one or more processors and memories. The memory is coupled to the processor and is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the spatial adaptive small sample learning method as described in the above embodiments.

[0105] The processor is used to control the overall operation of the small sample learning system to complete all or part of the steps of the above-mentioned spatial adaptive small sample learning method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support the operation of the small sample learning system, which can include, for example, instructions for any application or method operating on the small sample learning system, as well as application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, etc.

[0106] In one example embodiment, the small sample learning system can be implemented by a computer chip or entity, or by a product with certain functions, for executing the above-mentioned space-adaptive small sample learning method and achieving the technical effects consistent with the above-mentioned method. A typical embodiment is a computer, a server. Specifically, the computer can be, for example, a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] In another example embodiment, the present application also provides a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the space-adaptive small sample learning method in any one of the above-mentioned embodiments. For example, the computer-readable storage medium can be the above-mentioned memory including program instructions, which can be executed by the processor of the system to complete the above-mentioned space-adaptive small sample learning method and achieve the technical effects consistent with the above-mentioned method.

[0108] It should be noted that the above-mentioned embodiments are only illustrative. The technical solutions of various embodiments can be combined, and the order of various steps can be changed, all of which are within the protection scope of the present application.

[0109] The above-mentioned space-adaptive small sample learning method and system provided by the present application are described in detail. For those skilled in the art, any obvious modification made to it without departing from the essential content of the present application will constitute an infringement of the patent right of the present application and will bear the corresponding legal responsibility.

Claims

1. A spatially adaptive few-shot learning method, wherein the input data is an image, characterized in that... Includes the following steps: S1: Global feature extraction of images based on a pre-trained backbone network; S2: Based on contextual information in a specific category, learn the global relationship embedding between the query sample and the supporting category in a class-sensitive context to obtain the global similarity score between the query sample and the supporting class; The global similarity score is calculated using the following formula: ,in represents the global similarity score between the query sample q and the k-th supporting sample in the n-th class, where Ψ represents the similarity metric function. This indicates the hidden state in both directions. The components are then combined to form the final global relationship embedding. S3: Based on the global similarity score obtained in step S2, if the score is greater than or equal to the reliability threshold, the classification result of the query sample is obtained; if the similarity is less than the reliability threshold, proceed to step S4 to obtain the calibration classification result of the local stage. S4: Feed the information from the most similar categories of the given query example into the extended backbone network, recalculate the local similarity score between the query sample and the supporting classes using the local features of the query sample, and compare it with a local threshold to obtain the classification result of the query sample; the local similarity score Calculated using the following formula: ,in , To query the local features of a sample, express The L nearest neighbors among all local patches; the last layer of the pre-trained backbone network is a fully connected layer; the last layer of the extended backbone network is a spatially adaptive layer; S5: Adopt a task-oriented tuning strategy. For each task, the supporting features are averaged using a pre-trained backbone network. The normalized features are used to calculate the prototype representation of each class, and the prototype representation is used to initialize the task-specific classifier. Among them, the overall training function , For spatial adaptive loss function, Let be the loss function for the task-oriented tuning strategy, and ， ， Where T represents the number of query samples, N represents the number of supported categories, and K represents the number of supported samples.

2. The few-shot learning method based on spatial adaptation as described in claim 1, characterized in that: The spatial adaptive layer is a convolutional layer whose parameters are dynamically adjusted during training based on the input features to adapt to distinguishing local fine-grained details.

3. The few-shot learning method based on spatial adaptation as described in claim 2, characterized in that: The spatial adaptive layer takes the one-dimensional feature vector output by the fully connected layer as input through a convolution operation to obtain a two-dimensional feature map.

4. The few-shot learning method based on spatial adaptation as described in claim 3, characterized in that... For sample x, the extended backbone network extracts feature P that satisfies: ,in, It is the chosen ReLU activation function. This indicates an expansion of the backbone network.

5. The few-shot learning method based on spatial adaptation as described in claim 1, characterized in that: During the inference process, a task-oriented tuning strategy is adopted, which directly adjusts the parameters of the batch normalization layer in the pre-trained backbone for each task, and uses standard cross-entropy regularization to update the affine parameters of the batch normalization layer.

6. The few-shot learning method based on spatial adaptation as described in claim 1, characterized in that... It also includes the following steps: S6: After task-oriented optimization, calculate the relationships between the queried classes to distinguish their categories.

7. A spatially adaptive few-shot learning system, wherein the input data is an image, characterized in that... It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, which, when executed by the processor, causes the processor to implement the spatially adaptive few-shot learning method according to any one of claims 1 to 6.

Images