Feature cache optimization-based few-sample classification method research

Abstract

The invention discloses a few-sample classification method research based on feature cache optimization, and the method comprises the steps: (1), employing an improved ResNet-based SwAV model as a backbone, and enabling the SwAV model to be used for generating auxiliary features; (2) taking the image data and the text description as original input, and generating text input with rich downstream language semantics by utilizing GPT-3 to serve as text prompt of a CLIP model; (3) through a feature selection method based on feature similarity and difference, the problems of feature redundancy and inaccurate selection in the feature selection process are solved, feature dimensions with high selection value are identified, and normalization and enhancement zooming processing are performed on the features; and (4) utilizing visual contrast knowledge of SwAV, introducing a learnable cache model, and adaptively mixing prediction results from CLIP and SwAV. According to the method, under the CLIP framework without extra training, the accuracy of few-sample image classification is effectively improved through feature cache optimization and a self-adaptive hybrid strategy.

Description

Technical Field

[0001] The present invention relates to the field of multimodality, and particularly to a research on few-shot classification method based on feature cache optimization. Background Art

[0002] With the rapid development of deep learning and computer vision technologies, vision-language models (such as CLIP) have made remarkable breakthroughs in tasks such as image classification, object detection, and image generation. Especially in the field of few-shot learning, pre-trained models such as CLIP have demonstrated powerful cross-modal learning capabilities. By jointly training with a large number of image-text pairs, these models can effectively understand and generate semantic associations between images and texts. This feature enables CLIP to perform excellently in zero-shot learning tasks and can infer new tasks without additional labeled data.

[0003] In recent years, for the case of limited training sample sets, researchers have proposed various methods to improve the transfer learning ability of CLIP, namely few-shot learning based on CLIP. These methods are mainly divided into the following three categories:

[0004] (1) Prompt Tuning: By introducing learnable text prompts into the text encoder of CLIP, the adaptation ability of the model to downstream tasks is optimized.

[0005] (2) Adapter-based Tuning: Construct lightweight modules (such as multi-layer perceptron MLP) at the ends of the text and visual encoders to adjust the text and visual feature representations of downstream tasks.

[0006] (3) Cache-based Tuning: Propose a "soft" k-nearest neighbor classifier, store the visual features and labels of training samples, and combine them with zero-shot CLIP for final classification.

[0007] Although CLIP has achieved remarkable results in few-shot learning, there are still some key problems to be solved, especially how to further improve the accuracy and generalization ability of the model in the case of low data volume. One of the core challenges of few-shot learning is how to fully exploit and utilize existing knowledge and prior information when the training data is extremely limited. Although traditional few-shot learning methods (such as transfer learning, meta-learning, etc.) attempt to alleviate the data scarcity problem through different mechanisms, there are still the following limitations:

[0008] (1) The feature selection is not precise enough, resulting in feature redundancy or information loss;

[0009] (2) The task adaptability is insufficient and it is difficult to quickly adapt to the requirements of new tasks;

[0010] (3) The utilization of prior knowledge is insufficient, which limits the generalization ability of the model.

[0011] Therefore, a new technical solution is needed that can more effectively utilize multi-modal features and prior knowledge in the few-shot learning scenario to improve the classification performance and generalization ability of the model. Summary of the Invention

[0012] The object of the present invention is to propose a few-shot classification method based on feature cache optimization, aiming to solve the problems of insufficient accuracy and limited generalization ability in the existing few-shot classification tasks, so as to improve the classification accuracy and generalization performance of the few-shot classification tasks.

[0013] Technical Solution: To achieve the above object, the present invention provides a few-shot classification method based on feature cache optimization, and the specific implementation steps are as follows:

[0014] Step 1: Construct a deep learning model based on the improved ResNet architecture as the backbone network of the SwAV model to generate auxiliary features;

[0015] Step 2: Use the CLIP model to extract image features and corresponding text features. At the same time, use the SwAV model to extract auxiliary image features, and store the extracted features and the corresponding One-Hot encoded label vectors in the cache model, and the cache model adopts a key-value pair structure;

[0016] Step 3: Use the GPT-3 model to generate a text input (gpt3_prompt) with rich downstream task semantics, and convert the text input into text features through the text encoder of the CLIP model to align with the image features;

[0017] Step 4: Adopt a feature selection method based on feature similarity and difference, and screen out feature dimensions with high selection value by calculating the cosine similarity and difference scores between features;

[0018] Step 5: Perform L2 normalization on the feature dimensions identified in Step 4, and enhance and scale the features through a preset scaling factor to improve the discrimination ability of the features;

[0019] Step 6: Utilize the visual contrast learning knowledge of the SwAV model, combine the cache model constructed in Step 2, and dynamically adjust the contribution ratio of the prediction results of the CLIP model and the SwAV model through learnable weight parameters to generate the final classification result.

[0020] In Step 1, the specific process of the SwAV model for generating auxiliary features is as follows:

[0021] 1.1 Definition and Initialization of the Auxiliary Model

[0022] An auxiliary model is constructed based on the improved ResNet model architecture, which includes multiple residual blocks. Each residual block has an expansion factor and a width multiplier. The expansion factor is used to dynamically adjust the number of channels in the convolutional layers of the residual block, and the width multiplier is used to control the overall width of the model to adapt to input features of different scales. The construction process of the model is as follows:

[0023] First, define the residual block. The residual block includes three convolutional layers, a batch normalization layer, a ReLU activation function, and an optional downsampling module. Among them, the downsampling module is used to adjust the resolution of the feature map when needed. The residual block also includes a residual connection, which is used to add the input feature to the output feature of the convolutional layer to alleviate the vanishing gradient problem.

[0024] Then, construct the ResNet model. The ResNet model includes an initial convolutional layer, a max pooling layer, multiple residual layers, a global average pooling layer, and a fully connected layer. The initial convolutional layer is used to extract the primary features of the input image, with a convolutional kernel size of 7x7 and a stride of 2; the max pooling layer is used to further downsample the feature map, with a pooling kernel size of 3x3 and a stride of 2.

[0025] 1.2 Loading and Processing of Pre-trained Weights

[0026] First, load the pre-trained weights obtained by self-supervised learning. The self-supervised learning method adopts a contrastive learning strategy to train the model to extract general features by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs.

[0027] Further, adapt the loaded weights to ensure that the weights can be correctly loaded into the model. Finally, replace the fully connected layer of the model with an identity mapping, that is, directly output the input features to generate auxiliary features. The auxiliary features retain the high-level representation of the model for the input features and are applicable to various downstream tasks. In step 2, the specific process of extracting image features and their class labels is as follows:

[0028] 2.1 For a classification task with N classes and K samples per class, where each class contains K annotated images, denoted as X = x1, x2,..., x NK , labeled as Y = y1, y2,..., y NK , use the pre-trained CLIP model to extract image features F CLIP ∈R NK×C and the corresponding labels L CLIP ∈R NK×N, where C is the feature dimension. Meanwhile, use the SwAV model based on the improved ResNet architecture in step 1 to extract the auxiliary image feature F SwAV ∈R NK×C and the label L SwAV ∈R NK×N . The label C LIP and L SwAV are generated by performing One-Hot encoding on the true class of the image. Each label vector has a dimension of N (number of classes), and only the position corresponding to the class is 1, and the rest are 0. The specific formula is as follows:

[0029] F CLIP = VisualEncoder clip (X)

[0030] L CLIP = OneHot(Y)

[0031] F SwAV = SwAV(X)

[0032] L SwAV = OneHot(Y)

[0033] 2.2. The cache model adopts a key-value pair structure, where the key is the extracted image feature F CLIP and F SwAV , and the value is the corresponding One-Hot encoded label vector L CLIP and L SwAV .

[0034] In step 3, use GPT-3 to generate text prompts with rich downstream language semantics, and combine these prompts with class names through the CLIP model to generate text features for classification. The specific process is as follows:

[0035] 3.1. Use the GPT-3 model, with the given class name and predefined language template as input, to generate a text prompt gpt3_prompt with rich downstream task semantics.

[0036] 3.2. Further, for each class, input the generated multiple text prompts into the CLIP model respectively, and encode them into corresponding text embedding vectors; then average these text embedding vectors in the feature dimension. Then perform L2 normalization on the average embedding vectors of each class to ensure that the text feature vectors of each class have unit length, which is convenient for subsequent feature comparison and classification operations. Finally, generate a text feature matrix W ∈ R N×C , where N is the number of classes, C is the feature dimension, and each row of W corresponds to the normalized text embedding vector of a class, which is used to represent the class feature and serve as the input for subsequent few-shot classification tasks.

[0037] In step 4, the specific process of identifying the feature dimensions with high selection value is as follows:

[0038] 4.1. Input text features and image features, transpose the text features and reshape the image features to form a unified feature representation, ensuring the compatibility and consistency of text and image features in subsequent processing.

[0039] 4.2. Use cosine similarity to calculate the similarity between features and convert it into a similarity matrix by adjusting the distance formula. The specific formula is as follows:

[0040]

[0041] where cdist returns the distance between features, and by converting it into similarity, feats represents the features. 4.3. Introduce the SmoothL1 loss function to calculate the difference between features and the mean, and smooth the error to avoid the influence of extreme values. The specific formula is as follows:

[0042] criterion = -λ1·sim + λ2·SmoothL1(W)

[0043] where the SmoothL1 loss function is defined as:

[0044]

[0045] where x is the feature weight, μ is the mean of the feature weights, β is the smoothing parameter, and λ1 and λ2 are weight coefficients. 4.4. Perform weighted combination on the similarity matrix and the difference calculation result to form a comprehensive criterion for evaluating the quality of features.

[0046] 4.5. According to the comprehensive criterion value criterion, select the optimal feature subset through the Top-K selection algorithm. Introduce a caching mechanism to store the calculated similarity matrix sim, avoid repeated calculations, and improve the calculation efficiency. In step 5, the specific process of normalizing and enhancing the scaling of features is as follows:

[0047] Normalize the input feature vector feats ∈ R NK×C so that its L2 norm in the specified dimension is 1, and multiply the normalized feature vector by an enhancement factor ε to enhance the expression ability of the features. The specific formula is as follows:

[0048]

[0049] where feats represents the input feature vector, dim represents the dimension of normalization, ε is the enhancement factor used to adjust the amplitude of the features, ||·|| pDenotes the L2 norm (p = 2).

[0050] In step 6, the specific process of adaptively mixing the prediction results from CLIP and SwAV is as follows:

[0051] 6.1. For a few-shot classification task with N classes and K samples per class, first extract the text feature W ∈ R N×C and two visual features f CLIP ∈ R 1xC , f SwAV ∈ R 1×C , and use them as queries to retrieve various knowledge from the cache model. Then calculate the similarity between the visual features and the predefined class weights to obtain the initial zero-shot learning feature representation. The specific formula is as follows:

[0052] ζ zs = f CLIP W T

[0053] 6.2. Feature similarity calculation: Calculate the similarity between the CLIP visual feature f CLIP and the SwAV visual feature f SwAV and the cache features. This process can achieve the similarity measurement between features through matrix multiplication. The specific formula is as follows:

[0054]

[0055] 6.3. Based on the above similarity calculation, generate two predicted classification Logits: ζ CLIP , ζ SwAV ∈ R 1xN , which are the two keys of the CLIP cache model. The specific formula is as follows:

[0056] ζ CLIP = exp(-β(1 - χ CLIP ))L CLIP

[0057] ζ SwAV = exp(-β(1 - χ SwAV ))L SwAV

[0058] 6.4. Perform a weighted sum of the fused feature representation and the initial zero-shot learning feature representation to obtain the final feature representation. The specific formula is as follows:

[0059] Logits = ζ zs + α(ζ zs ⊙ [ζ CLIP , ζ SwAV )

[0060] where Logits ∈ R CK×C , α and β are hyperparameters, and ⊙ represents the Hadamard inner product.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. The present invention combines the multi-modal contrastive learning ability of the CLIP model and the self-supervised learning mechanism of the SwAV model. By adaptively mixing the prediction results from CLIP and SwAV, the model's ability to fuse image and text features is enhanced. This method not only retains the zero-shot classification ability of CLIP but also further improves the robustness and generalization ability of the features through the visual contrast knowledge of SwAV.

[0063] 2. The present invention solves the problems of feature redundancy and inaccurate selection in traditional methods through a feature selection method based on feature similarity and difference. Through normalization and enhanced scaling processing, the feature dimensions with high selection value are identified, thereby improving the discrimination ability of the features. This method performs well in few-shot learning scenarios and can effectively utilize limited sample data to improve classification performance.

[0064] 3. The present invention introduces a learnable cache model to adaptively adjust the weights of the prediction results of CLIP and SwAV, further optimizing the classification performance. This adaptive mechanism can dynamically adjust the behavior of the model according to the characteristics of the input data, thereby maintaining stable performance output in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a design diagram of the overall framework for the research on the few-shot classification method based on feature caching optimization DETAILED DESCRIPTION OF THE INVENTION

[0066] The present invention will be described in detail below with reference to the drawings and embodiments. It should be noted that the following embodiments are only used to help understand the technical solution of the present invention and do not constitute any limitation to the protection scope of the present invention. Figure 1 It is the main framework design diagram of the technical solution of the present invention. As Figure 1 shown, the present invention proposes a few-shot classification method based on feature caching optimization, which mainly includes a visual feature module, a feature selection module, and a similarity calculation module. The specific implementation steps of each module are described in detail below. Step 1, the visual feature module inputs visual features and text features for the few-shot classification task, and the basic steps are as follows:

[0067] 1.1. Use an improved ResNet model architecture as the backbone of the SwAV model to generate auxiliary features.

[0068] (1) Define the residual block: The residual block includes three convolutional layers, a batch normalization layer, a ReLU activation function, and an optional downsampling module. Among them, the downsampling module is used to adjust the resolution of the feature map when needed. The residual block also includes a residual connection, which is used to add the input feature to the output feature of the convolutional layer to alleviate the vanishing gradient problem.

[0069] (2) Construct the ResNet model: The ResNet model includes an initial convolutional layer, a max pooling layer, multiple residual layers, a global average pooling layer, and a fully connected layer. The initial convolutional layer is used to extract the primary features of the input image, with a convolutional kernel size of 7x7 and a stride of 2; the max pooling layer is used to further downsample the feature map, with a pooling kernel size of 3x3 and a stride of 2.

[0070] (3) Then load the model weights pre-trained by the SwAV self-supervised learning method to ensure that the model has good feature extraction capabilities. Next, perform an adaptation process on the loaded weights to ensure that the weights can be correctly loaded into the model.

[0071] (4) Finally, replace the fully connected layer of the ResNet model with an identity mapping to generate auxiliary features, which are used for subsequent feature fusion or task-specific processing.

[0072] 1-2. Use ResNet-50 as the visual encoder and use the transformer as the text encoder to construct the backbone of the CLIP model for obtaining image features. At the same time, combine the SwAV model backbone described in 1-1 to generate auxiliary features. The specific formula is as follows:

[0073] F CLIP =VisualEncoder clip (X)

[0074] L CLIP =OneHot(Y)

[0075] F SwAV =SwAV(X)

[0076] L SwAV =OneHot(Y)

[0077] 1-3. Consider the features F CLIP extracted by the CLIP model and the features F SwAV generated by the SwAV model as keys respectively, and store the corresponding One-Hot encoded label vectors L CLIP and L SwAV as values in the cache model.

[0078] 1-4. For N classes, a series of unified language templates are introduced. These templates include but are not limited to the following:

[0079] “Whata [CLASS] lookslike?”

[0080] “Howtorecognizea [CLASS]?”

[0081] “[CLASS]:Imagecaption.”

[0082] Among them, [CLASS] is a class placeholder, which will be replaced by the specific class name during actual input. Input the above language templates into the GPT-3 model to generate language prompts for each class. The generated set of language prompts is denoted as T N =T1,T2,...,T N , where Ti represents the language prompt corresponding to the i-th class.

[0083] Mark the text prompts in T N as the input of the CLIP text encoder to obtain the text feature W. It can be implemented by the following formula:

[0084] W=CLIP text (T N ) T

[0085] where CLIP text represents the text encoder of the CLIP model.

[0086] Step 2. The feature selection module screens out the feature dimensions with high selection value for few-shot classification. The basic steps are as follows:

[0087] 2-1. Input the text features and image features obtained in Step 1, transpose the text features, and reshape the image features to align their dimensions with the text features to form a unified feature representation.

[0088] 2-2. Use the cosine similarity to calculate the similarity between features, and convert it into a similarity matrix by adjusting the distance formula. The specific formula is as follows:

[0089]

[0090] Among them, cdist returns the distance between features. By converting it into similarity, feats represents the features. 2-3. Introduce the SmoothL1 loss function to calculate the difference between the features and the mean, and smooth the error to avoid the influence of extreme values. The specific formula is as follows:

[0091] criterion = -λ1·sim + λ2·SmoothL1(W)

[0092] where W is the text feature weight, and λ1 and λ2 are weight coefficients.

[0093] According to the criterion value criterion, the optimal features are selected through the following formula:

[0094] indices = TopK(criterion, k)

[0095] where TopK(·) represents selecting the top k optimal features, and k is the number of features selected.

[0096] 2 - 4. The specific process of normalizing and enhancing the scaling of the features is as follows:

[0097] The input feature vector is normalized so that its L2 norm on the specified dimension dim is 1, and the normalized feature vector is multiplied by an enhancement factor ε to improve the expression ability of the features. The specific formula is as follows:

[0098]

[0099] where feats represents the input feature vector, dim represents the dimension of normalization, ε is the enhancement factor used to adjust the amplitude of the features, and ||·|| p represents the L2 norm (p = 2).

[0100] Step 3. The similarity calculation module is used to fuse the prediction results of the SwAV model and the CLIP model during the few - shot classification process, thereby improving the classification performance. The specific steps are as follows:

[0101] 3 - 1. First, extract its two visual features f CLIP , f SwAV , and use them as queries to retrieve the relevant knowledge features from the cache model. Then, calculate the similarity between the visual features and the cache features, as well as the similarity between the visual features and the SwAV cache auxiliary features. The similarity calculation is achieved through matrix multiplication. The specific formula is as follows:

[0102]

[0103] 3 - 2. Based on the above similarity calculation, generate three predicted classification Logits: ζ zs , ζ CLIP , ζ SwAV ∈R 1xN , which are respectively the zero - shot alignment Logits of CLIP and the two keys of the cache model. The specific formula is as follows:

[0104] ζzs = f CLIP W T

[0105] ζ CLIP = exp(-β(1 - χ CLIP ))L CLIP

[0106] ζ SwAV = exp(-β(1 - χ SwAV ))L SwAV

[0107] Wherein, W represents the text feature obtained by encoding with the CLIP text encoder, and exp(-β·(1 - x)) is used as a non-linear modulator to control the sharpness of the similarity matrix.

[0108] 3-3. Perform weighted summation on the fused feature representation and the initial zero-shot learning feature representation to obtain the final feature representation. The specific formula is as follows:

[0109] Logits = ζ zs + α(ζ zs ⊙ [ζ CLIP , ζ SwAV )

[0110] Where Logits ∈ R CK×C , represents the final classification Logits matrix, C represents the number of categories, K represents the number of samples in each category, and α is a hyperparameter used to adjust the contribution weights of the CLIP model and the SwAV model. The above embodiments have described in detail the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, supplements, and equivalent replacements made within the principle scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A research on a few-sample classification method based on feature cache optimization, characterized in that: The following steps are involved: Step 1: Build a deep learning model based on the improved ResNet architecture as the backbone network of the SwAV model to generate auxiliary features; Step 2: The CLIP model is used to extract image features and corresponding text features. At the same time, the SwAV model is used to extract auxiliary image features, and the extracted features and the corresponding One-Hot encoded label vectors are stored in a cache model, which adopts a key-value pair structure. Step 3: Use the GPT-3 model to generate a text input gpt3_prompt with rich downstream task semantics, and convert the text input into text features through the text encoder of the CLIP model, and align them with the image features; Step 4: Use a feature selection method based on feature similarity and difference to screen out feature dimensions with higher selection value by calculating the cosine similarity and difference scores between features; Step 5: Perform L2 normalization on the feature dimensions identified in step 4, and enhance and scale the features by a preset scaling factor to improve the distinguishing ability of the features; Step 6: Using the visual contrast learning knowledge of the SwAV model and the cache model constructed in step 2, the contribution ratio of the prediction results of the CLIP model and the SwAV model is dynamically adjusted through learnable weight parameters to generate the final classification result.

2. According to claim 1, a research on a few-sample classification method based on feature cache optimization is characterized in that: The step 1 specifically comprises the following steps: Step 1.1: Definition and initialization of auxiliary models An auxiliary model is constructed based on an improved ResNet model architecture, which includes multiple residual blocks. Each residual block has an expansion factor and a width multiplier. The expansion factor is used to dynamically adjust the number of channels of the convolutional layer in the residual block, and the width multiplier is used to control the overall width of the model to adapt to input features of different scales. The construction process of the model is as follows: (1) Define a residual block: The residual block includes three convolutional layers, a batch normalization layer, a ReLU activation function, and an optional downsampling module. The downsampling module is used to adjust the resolution of the feature map when necessary. The residual block also includes a residual connection for adding the input features to the convolutional layer output features to alleviate the gradient vanishing problem. (2) Constructing a ResNet model: The ResNet model includes an initial convolution layer, a maximum pooling layer, multiple residual layers, a global average pooling layer, and a fully connected layer. The initial convolution layer is used to extract primary features of the input image, with a convolution kernel size of 7x7 and a step size of 2; the maximum pooling layer is used to further downsample the feature map, with a pooling kernel size of 3x3 and a step size of 2. Step 1.2: Loading and processing pre-trained model weights (1) Loading pre-trained weights trained by a self-supervised learning method, wherein the self-supervised learning method adopts a contrastive learning strategy to train the model to extract common features by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. (2) Adapt the loaded weights to ensure that the weights can be correctly loaded into the model. (3) The fully connected layer of the model is replaced with an identity mapping, that is, the input features are directly output to generate auxiliary features. The auxiliary features retain the model's high-level representation of the input features and are suitable for a variety of downstream tasks.

3. According to claim 1, a research on a few-sample classification method based on feature cache optimization is characterized in that: The step 2 is specifically as follows: Step 2.1: Extract image features and labels For a classification task with N categories and K samples per category, each category contains K annotated images, denoted by X = {x1, x2, ..., x NK }, marked as Y = {y1,y2,...,y NK }, use the pre-trained CLIP model to extract image features F CLIP ∈R NK×C and the corresponding label L CLIP ∈R NK×N , where C is the feature dimension. At the same time, the auxiliary image feature F is extracted using the SwAV model with improved ResNet as the backbone SwAV ∈R NK×C and label L SwAV ∈R NK×N The label L CLIP and L SwAV It is generated by One-Hot encoding the real category of the image, where the dimension of each label vector is N (the number of categories), and only the position corresponding to the category is 1, and the rest of the positions are 0. This can be achieved by the following formula: F CLIP =VisualEncoder clip (X) L CLIP =OneHot(Y) F SwAV =SwAV(X) L SwAV =OneHot(Y) Step 2.2: Build a cache model The cache model adopts a key-value pair structure, where the key is the extracted image feature F CLIP and F SwAV , the value is the corresponding One-Hot encoded label vector L CLIP and L SwAV .

4. According to claim 1, a research on a few-sample classification method based on feature cache optimization is characterized in that: The step 3 is specifically as follows: Step 3.

1. Use the GPT-3 model to generate a text prompt gpt3_prompt with rich downstream task semantics, taking the given category name and predefined language template as input. Step 3.2: For each category, the generated multiple text prompts are input into the CLIP model and encoded into corresponding text embedding vectors; these text embedding vectors are then averaged in the feature dimension. The average embedding vector of each category is then L2 normalized to ensure that the text feature vector of each category has unit length, which is convenient for subsequent feature comparison and classification operations. Finally, the text feature matrix W∈R is generated. N×C , where N is the number of categories, C is the feature dimension, and each row of W corresponds to a normalized text embedding vector of a category, which is used to represent the category features and serve as the input for subsequent few-shot classification tasks.

5. According to claim 1, a research on a few-sample classification method based on feature cache optimization is characterized in that: The step 4 is specifically as follows: Step 4.

1. Input text features W∈R N×C And image features F∈R NK×C , the text features are transposed and the image features are reshaped to form a unified feature representation. Step 4.2: Use cosine similarity to calculate the similarity between features and convert it into a similarity matrix sim∈R by adjusting the distance formula NK×C . This can be achieved using the following formula: Among them, cdist returns the distance between features, which is converted into similarity, and feats represents the features. Step 4.3: Introduce the Smooth L1 loss function to calculate the difference between the feature and the mean, and smooth the error to avoid the influence of extreme values. The specific formula is as follows: criterion=-λ1·sim+λ2·SmoothL1(W) Among them, the SmoothL1 loss function is defined as: Where x is the feature weight, μ is the mean of the feature weight, β is the smoothing parameter, and λ1 and λ2 are weight coefficients. Step 4.4: Perform a weighted combination of the similarity matrix and the difference calculation results to form a comprehensive criterion for evaluating the quality of features. Step 4.5: According to the comprehensive criterion value, select the optimal feature subset through the Top-K selection algorithm. Step 4.6: Introduce a cache mechanism to store the calculated similarity matrix sim to avoid repeated calculations and improve calculation efficiency.

6. According to claim 1, a research on a few-sample classification method based on feature cache optimization is characterized in that: The step 5 is specifically as follows: For the input feature vector feats∈R NK×C Normalize it so that its L2 norm in the specified dimension is 1, and multiply the normalized feature vector by an enhancement factor ε to improve the expressiveness of the feature. The specific formula is as follows: Among them, feats represents the input feature vector, dim represents the normalized dimension, ε is the enhancement factor used to adjust the amplitude of the feature, ||·|| p represents the L2 norm (p=2).

7. The research on a few-sample classification method based on feature cache optimization according to claim 1 is characterized in that: The step 6 is specifically as follows: Step 6.1: For a few-sample classification task with N classes and K samples per class, first extract the text feature W∈R N×C and two visual features f CLIP ∈R 1xC ,f SwAV ∈R 1×C , and use it as a query to retrieve relevant knowledge from the cache model. Then calculate the similarity between the visual features and the predefined category weights to obtain the initial zero-shot learning feature representation. The specific formula is as follows: g zs =f CLIP W T Step 6.2: Calculate visual features f CLIP and f SwAV The similarity between the cached features and the auxiliary features of the SwAV cache is measured by matrix multiplication. The specific formula is as follows. Step 6.3, calculate the cache logits of CLIP and SwAV, the specific formula is as follows: g CLIP =exp(-β(1-x CLIP ))L CLIP g SwAV =exp(-β(1-x SwAV ))L SwAV Among them, exp(-β·(1-x)) is used as a nonlinear modulator to control the sharpness of the similarity matrix. Step 6.4: Perform weighted summation of the fused feature representation and the initial zero-shot learning feature representation to obtain the final feature representation. The specific formula is as follows: Logits=ζ zs +a(g zs ⊙[ζ CLIP ,g SwAV ]) where Logits∈R CK×C , α and β are hyperparameters, and ⊙ represents the Hadamard inner product.

Images