Cross-domain few-sample image recognition method, system and equipment for training during testing

Abstract

The invention discloses a cross-domain few-sample image recognition method, system and equipment for training during testing, and the method comprises the steps: firstly extracting the rotating image features of a source domain training set image through a first feature extractor, training a self-supervised learning model, and obtaining an optimized self-supervised learning model; then, a second feature extractor is adopted to extract rotating image features of the target domain support set image as input of the optimized self-supervised learning model and the nearest centroid classifier, combination loss is obtained, and parameters of the second feature extractor are optimized by minimizing the combination loss; and obtaining an optimized second feature extractor and a trained image recognition model, extracting features of the target domain query set image by using the optimized second feature extractor as input of a nearest centroid classifier, and performing recognition test on the image recognition model according to the accuracy of the nearest centroid classifier to obtain an image recognition result. According to the invention, the accuracy of cross-domain image recognition with few samples can be improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, system, and device for cross-domain few-shot image recognition trained during testing. Background Technology

[0002] In the fields of computer vision and pattern recognition, deep learning-based image classification techniques have made significant progress in scenarios with large-scale labeled data. However, the labeling cost in such scenarios is extremely high, and image data from different domains have different feature distributions, resulting in significant cross-domain distribution shifts. To address this problem of cross-domain image classification and recognition with limited labeled data, the current mainstream solution is meta-learning techniques.

[0003] Meta-learning is a machine learning technique that aims to enable models to "learn how to learn." This method mimics human learning strategies, first performing pre-training on a source dataset that does not overlap with the target test dataset (the meta-training phase). This allows the model to quickly simulate learning different tasks and acquire sufficient prior knowledge when faced with limited labeled data. After learning on the source domain, the meta-learning model enters the meta-testing phase, applying the prior knowledge to the target task. Using a small number of labeled samples in the target domain, it constructs a classification task to classify and recognize the target image data.

[0004] However, in cross-domain scenarios, changes in the feature distribution of data can make it difficult for generalized meta-learning methods to generalize to new domains. To address this issue, most meta-learning methods for cross-domain few-shot problems choose to generalize the source domain data during the meta-training phase to align with the feature distribution changes of the target domain. Therefore, these methods suffer from problems such as ignoring the semantic nature of features, relying on source domain data to enhance alignment with the target domain, and insufficient application of self-supervised learning in few-shot scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a cross-domain few-shot image recognition method, system, and device that is trained during testing, which can make full use of the semantic features of the target data to perform classification and recognition of approximate source domains, and achieve robust classification and recognition under cross-domain few-shot conditions.

[0006] To achieve the above objectives, the present invention provides a cross-domain few-shot image recognition method trained during testing, comprising:

[0007] The first feature extractor is used to extract rotational image features from the source domain training set images, and the self-supervised learning model is trained to obtain the optimized self-supervised learning model.

[0008] The parameters of the first feature extractor and the optimized self-supervised learning model are frozen. The second feature extractor extracts rotational image features from the target domain support set image as input to the optimized self-supervised learning model, resulting in a rotation angle classification loss. The mean of each type of rotational image feature is calculated as the nearest centroid classifier, and the rotational image features from the target domain support set image are used as input to the nearest centroid classifier, resulting in a classification loss. The parameters of the second feature extractor are optimized by minimizing the combined loss, resulting in an optimized second feature extractor and a trained image recognition model. The combined loss includes the classification loss and the rotation angle classification loss.

[0009] The optimized second feature extractor is used to extract features from the target domain query set image, which are then used as input to the nearest centroid classifier. The image recognition model is then tested based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

[0010] Optionally, the step of using a first feature extractor to extract rotational image features from the source domain training set images, and training the self-supervised learning model to obtain an optimized self-supervised learning model includes:

[0011] The first feature extractor is used to extract rotational image features from the source domain training set images;

[0012] The rotation image features of the source domain training set images are used as input to the self-supervised learning model. The parameters of the first feature extractor are frozen, and the parameters of the self-supervised learning model are optimized by minimizing the rotation angle classification loss, resulting in an optimized self-supervised learning model.

[0013] Optionally, the step of extracting rotational image features from the source domain training set images using a first feature extractor includes:

[0014] The source domain training set images are preprocessed by rotation operators to generate rotated images with four rotation angles, thus obtaining rotated images of the source domain training set images; wherein the rotation angles include 0°, 90°, 180° and 270°.

[0015] The first feature extractor is used to extract features from the rotated images of the source domain training set images to obtain the rotated image features of the source domain training set images.

[0016] Optionally, the rotation angle classification loss is:

[0017]

[0018] in, g is the total number of samples in the source domain training set images. θ f represents a self-supervised learning model.φ This represents the first feature extractor. Represents the input image X in the training set. m The rotated image at the k-th rotation angle, This represents the predicted probability for the k-th rotation angle.

[0019] Optionally, after taking the rotation image features of the source domain training set images as input to the self-supervised learning model, freezing the parameters of the first feature extractor, and optimizing the parameters of the self-supervised learning model by minimizing the rotation angle classification loss to obtain an optimized self-supervised learning model, the method further includes:

[0020] The rotation image features extracted from the source domain validation set images by the first feature extractor are used as input to the optimized self-supervised learning model. The hyperparameters of the optimized self-supervised learning model are then optimized based on the accuracy of the optimized self-supervised learning model until the hyperparameters of the optimized self-supervised learning model meet the preset hyperparameter conditions of the self-supervised learning model, thus obtaining the final self-supervised learning model.

[0021] Optionally, the objective function for minimizing the combination loss is:

[0022]

[0023] Where α represents the task-specific adapter A α The parameter N; S S represents the total number of samples in the target domain support set of images; j X represents the target domain support set; rot S represents j The corresponding rotated image input; y represents the category label corresponding to the image in the target domain support set; y ssl The L represents the rotation angle category label corresponding to the rotated image of the target domain support set image; ce The classification loss is represented by L, calculated using the nearest centroid classifier. ssl λ is the rotation angle classification loss, calculated using the optimized self-supervised learning model; f is the weight balancing the two losses; φ,α This refers to the second feature extractor.

[0024] Optionally, the first feature extractor is obtained by supervised learning training on a large-scale dataset.

[0025] To achieve the above objectives, the present invention also provides a cross-domain few-shot image recognition system trained during testing, comprising:

[0026] The self-supervised learning model training module is used to extract rotation image features from the source domain training set images using the first feature extractor, train the self-supervised learning model, and obtain the optimized self-supervised learning model.

[0027] The feature extractor training module is used to freeze the parameters of the first feature extractor and the optimized self-supervised learning model. It uses a second feature extractor to extract rotational image features from the target domain support set image as input to the optimized self-supervised learning model, obtaining a rotation angle classification loss. It calculates the mean of each type of rotational image feature as the nearest centroid classifier, using the rotational image features from the target domain support set image as input to the nearest centroid classifier, obtaining a classification loss. It optimizes the parameters of the second feature extractor by minimizing the combined loss, obtaining the optimized second feature extractor and the trained image recognition model. The combined loss includes the classification loss and the rotation angle classification loss.

[0028] The image recognition testing module is used to extract features from the target domain query set image using the optimized second feature extractor as input to the nearest centroid classifier, and to perform recognition testing on the image recognition model based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

[0029] To achieve the above objectives, the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the cross-domain few-shot image recognition method trained during testing as described above.

[0030] Compared with existing technologies, this invention provides a test-time trained cross-domain few-shot image recognition method, system, and device. The training method first forces the image recognition model to learn semantically invariant features of the image, rather than surface visual differences, through a rotation prediction task of a self-supervised learning model. This enables the image recognition model to establish a semantic bridge between the source and target domains, effectively addressing the domain shift problem. Then, by introducing a dual-loss co-optimization of the image recognition model, the classification loss ensures the discriminativeness of features through the nearest centroid classifier, maintaining the separability of features of different categories in the embedding space, which can improve the few-shot classification accuracy. The rotation angle classification loss utilizes the semantic constraints of the self-supervised learning model, forcing the feature extractor to focus on the intrinsic structure of the image rather than surface changes, enhancing cross-domain generalization ability. The combined optimization of the dual losses can balance the discriminativeness of features and semantic consistency, avoiding the model from falling into overfitting or domain shift traps. The image recognition model trained by the embodiments of this invention can fully utilize the semantic features of the target data to perform near-source domain classification and recognition, achieving robust classification and recognition under cross-domain few-shot conditions. Attached Figure Description

[0031] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of a cross-domain few-shot image recognition method trained during testing, provided by an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the image rotation preprocessing flow provided in an embodiment of the present invention;

[0034] Figure 3 This is a parameterized schematic diagram of a task-specific adapter provided in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of the feature extractor of the introduction adapter provided in an embodiment of the present invention;

[0036] Figure 5 This is a structural block diagram of a cross-domain few-shot image recognition system trained during testing, provided in an embodiment of the present invention.

[0037] Figure 6 This is a structural block diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0039] See Figure 1 , Figure 1 This is a flowchart of a cross-domain few-shot image recognition method trained during testing, provided by an embodiment of the present invention. The method includes steps S1 to S3:

[0040] It should be noted that, in this embodiment of the invention, the cross-domain few-shot image recognition method trained during testing includes a meta-training stage and a meta-testing stage. The meta-training stage includes step S1, and the meta-testing stage includes steps S2 to S3.

[0041] S1. The first feature extractor is used to extract the rotation image features of the source domain training set images, and the self-supervised learning model is trained to obtain the optimized self-supervised learning model.

[0042] It should be noted that the source domain refers to the domain with a large amount of labeled data, used for model pre-training (meta-training stage) to learn general features or semantic information. In the meta-training stage of this invention, the dataset needs to be divided in the source domain using the training-validation splitting method to obtain the training set and validation set. The target domain refers to the domain where the model needs to perform actual recognition; it is usually characterized by a small amount of data and scarce labels (few-sample scenarios), and its categories do not overlap with those of the source domain. In the meta-testing stage of this invention, a subset is divided from the target domain data for each few-sample task. The subset containing a small number of labeled samples is the support set, used to guide the model to adapt to new tasks in the meta-testing stage. The subset containing unlabeled samples is the query set, used to evaluate the model's classification performance.

[0043] For example, the Mini-Imagenet dataset can be used as the source domain; the BSCD-FSL dataset can be used as the target domain, including data from the Cropdiseases, EuroSAT, ISIC, and ChestX datasets.

[0044] Preferably, the first feature extractor is trained through supervised learning on a large-scale dataset (such as ImageNet). For example, the first feature extractor may employ a ResNet18 network.

[0045] In one optional embodiment, step S1 specifically includes:

[0046] S101. Use the first feature extractor to extract the rotation image features of the source domain training set images;

[0047] S102. The rotation image features of the source domain training set images are used as input to the self-supervised learning model. The parameters of the first feature extractor are frozen, and the parameters of the self-supervised learning model are optimized by minimizing the rotation angle classification loss to obtain the optimized self-supervised learning model.

[0048] The step of extracting rotational image features from the source domain training set images using a first feature extractor includes:

[0049] The source domain training set images are preprocessed by rotation operators to generate rotated images with four rotation angles, thus obtaining rotated images of the source domain training set images; wherein the rotation angles include 0°, 90°, 180° and 270°.

[0050] The first feature extractor is used to extract features from the rotated images of the source domain training set images to obtain the rotated image features of the source domain training set images.

[0051] It should be noted that the rotation preprocessing of images in each dataset in subsequent embodiments of the present invention adopts a similar processing method, which will not be repeated hereafter.

[0052] It should be noted that the self-supervised learning model in this embodiment of the invention is tasked with predicting the rotation angle of the input image. Therefore, the parameters of the self-supervised learning model are mainly optimized by minimizing the rotation angle classification loss.

[0053] Specifically, the rotation angle classification loss is:

[0054]

[0055] in, g is the total number of samples in the source domain training set images. θ f represents a self-supervised learning model. φ This represents the first feature extractor. Represents the input image X in the training set. m The rotated image at the k-th rotation angle, This represents the predicted probability for the k-th rotation angle.

[0056] In an optional embodiment, after step S102, the method further includes:

[0057] S103. The rotation image features of the source domain validation set image are extracted using the first feature extractor and used as the input of the optimized self-supervised learning model. The hyperparameters of the optimized self-supervised learning model are optimized according to the accuracy of the optimized self-supervised learning model until the hyperparameters of the optimized self-supervised learning model meet the preset hyperparameter conditions of the self-supervised learning model, so as to obtain the final self-supervised learning model.

[0058] S2. Freeze the parameters of the first feature extractor and the optimized self-supervised learning model. Use the second feature extractor to extract the rotation image features of the target domain support set image as the input of the optimized self-supervised learning model to obtain the rotation angle classification loss. Calculate the mean of each type of rotation image feature as the nearest centroid classifier. Use the rotation image features of the target domain support set image as the input of the nearest centroid classifier to obtain the classification loss. Optimize the parameters of the second feature extractor by minimizing the combined loss to obtain the optimized second feature extractor and the trained image recognition model. The combined loss includes the classification loss and the rotation angle classification loss.

[0059] It should be noted that the trained image recognition model includes a first feature extractor, an optimized self-supervised learning model, an optimized second feature extractor, and a nearest centroid classifier;

[0060] In one alternative embodiment, the second feature extractor is obtained by connecting the residual of the first feature extractor to a task-specific adapter, the task-specific adapter being parameterized by specific parameters;

[0061] The residual connection method is as follows:

[0062]

[0063] Where X is the input tensor, f φl f represents the first feature extractor φ The l-th layer has a weight of φl and A α A represents the task-specific adapter obtained by parameterizing parameter α. α (X) = X*α, where * represents the convolution operation.

[0064] In one alternative embodiment, the objective function for minimizing the combination loss is:

[0065]

[0066] Where α represents the task-specific adapter A α The parameter N; S S represents the total number of samples in the target domain support set of images; j X represents the target domain support set; rot S represents j The corresponding rotated image input; y represents the category label corresponding to the image in the target domain support set; y ssl The L represents the rotation angle category label corresponding to the rotated image of the target domain support set image; ce The classification loss is represented by L, calculated using the nearest centroid classifier. ssl λ is the rotation angle classification loss, calculated using the optimized self-supervised learning model; f is the weight balancing the two losses; φ,α This refers to the second feature extractor.

[0067] Therefore, after freezing the parameters of the first feature extractor, optimizing the parameters of the second feature extractor by minimizing the combined loss is essentially optimizing only the parameters of the task-specific adapter.

[0068] S3. Use the optimized second feature extractor to extract features from the target domain query set image as input to the nearest centroid classifier, and perform recognition tests on the image recognition model based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

[0069] In summary, the test-time trained cross-domain few-shot image recognition method provided by this invention is applied to an image recognition model, which includes a first feature extractor, a self-supervised learning model, a second feature extractor, and a nearest centroid classifier. The method first forces the image recognition model to learn semantically invariant features of the image, rather than surface visual differences, through a rotation prediction task of the self-supervised learning model. This enables the image recognition model to establish a semantic bridge between the source and target domains, effectively addressing the domain shift problem. Then, by introducing a dual-loss co-optimization of the image recognition model, the classification loss ensures the discriminativeness of features through the nearest centroid classifier, maintaining the separability of features of different categories in the embedding space, thus improving few-shot classification accuracy. The rotation angle classification loss utilizes the semantic constraints of the self-supervised learning model, forcing the feature extractor to focus on the image's intrinsic structure rather than surface changes, enhancing cross-domain generalization ability. The combined optimization of the dual losses balances the discriminativeness of features with semantic consistency, avoiding overfitting or domain shift traps. Image recognition using the image recognition model trained by this invention can fully utilize the semantic features of the target data for near-source domain classification, achieving robust classification under cross-domain few-shot conditions.

[0070] To enable those skilled in the art to better understand the specific implementation process of the present invention, the cross-domain few-sample image recognition method provided in the embodiments of the present invention will be described in more detail below.

[0071] First, in the meta-training phase, this invention first uses the training-validation splitting method to partition the dataset in the source domain, as follows:

[0072] Assuming source domain y i ∈C s , where X i Let y represent the feature of the i-th sample. i C is the category label corresponding to the i-th sample. s This represents the set of basic categories, containing a large number of labeled samples, where N1 is the total number of samples in the source domain.

[0073] Preferably, in this embodiment of the invention, the samples in the source domain are divided into a training set at a ratio of 7:3. and verification set in and These are the number of samples in the training set and the validation set, respectively.

[0074] Then, in the meta-testing phase, a large number of target tasks need to be constructed in the target domain. The image recognition model uses the feature representations learned on the support set to accurately classify and predict samples in the query set, thereby evaluating the model's generalization ability on new categories. Specifically:

[0075] Assuming target domain y i ∈C t , where X i Let y represent the feature of the i-th sample. i C is the category label corresponding to the i-th sample. t It is a new set of categories, each with only a small number of labeled samples, and satisfying C s ∩C t =φ, which means that the base category and the new category are disjoint sets, and N2 is the total number of samples in the target domain.

[0076] Assume T is the target domain D t The small sample task set constructed in the middle, where T j ={S j Q j} represents the j-th few-sample task, containing a support set S. j and a query set Q j Among them, from the new category set C t C categories are randomly selected from the samples to form the sample task T. j Category set For each category From the target domain D t K labeled samples are randomly selected from the sample to form the support set S. j ,Right now Where N S =C×K. For each category From the target domain D t M unlabeled samples are randomly selected from the dataset to form the query set Q. j ,Right now Where N Q = C × M. It's important to note that the query set Q... j The samples in the meta-test phase are unlabeled test samples, and their true labels y are hidden, used only to evaluate the predictive performance of the model.

[0077] Further, see Figures 2-4 In the meta-training phase, a training-validation split is used to divide the training and validation sets for training, constructing a self-supervised learning model. The steps are as follows:

[0078] Step (1): As Figure 2 As shown, a rotation operator R is first defined. β , where β represents the rotation angle. For the training set Each input image X m Its rotated image is generated at four fixed angles: 0°, 90°, 180°, and 270°, denoted as [missing characters]. Where k∈{1,2,3,4} corresponds to these angles respectively.

[0079] Step (2): Rotate the generated image Input to the pre-trained feature extractor f φ In the process, feature representations of each rotated image are obtained.

[0080] Step (3): Represent the features of the rotated image Input to a self-supervised learning (SSL) model g θ In the middle, g θ Output a 4-dimensional probability distribution. Represents image X m The probability of being rotated to any of these four angles.

[0081] Step (4): Freeze the pre-trained feature extractor f φ The parameter φ is only optimized for the SSL model g. θ The parameter θ is determined, and parameter optimization is achieved by minimizing the SSL loss, i.e.:

[0082] θ * =arg min θ L ssl (1)

[0083] The optimal SSL parameters θ are obtained by combining them with the fixed pre-trained feature extractor parameters φ. * .

[0084] Step (5): Use the validation set To evaluate the performance of the optimized SSL model. The validation set Image X in n Generate the corresponding rotated image using the same method as in step (1). And obtain its feature representation. Optimize the SSL model by calculating the accuracy on the validation set. Hyperparameters.

[0085] Step (6): Based on the results of the validation set, adjust the hyperparameters (such as learning rate, regularization term, etc.), and repeat steps (3) to (5) to further optimize the performance of the SSL model, ensuring that the SSL model performs well on both the training and validation sets. Finally, the optimized SSL model is obtained as the feature measurement model.

[0086] In the meta-testing phase, tasks T from a small sample task set within the target domain are used. j ={S j Q j}∈{T j The model evaluation process is as follows:

[0087] Step (7): For each task T j For support set S j The image in the image is preprocessed in a similar manner to step (1) to generate the corresponding rotated image.

[0088] Step (8): Freeze the pre-trained feature extractor f φ The parameters φ and the optimal SSL parameters θ obtained during the meta-training phase. * To leverage the feature metric model built on the source domain

[0089] Step (9): In the feature extractor f φ Introducing a task-specific adapter A α The feature extractor f is obtained after adapter adjustment. φ,α .

[0090] Specifically, the f φ,α To integrate with the feature extractor of the task-specific adapter, the connection type is residual connection ⊕, and the parameterization is matrix multiplication. The residual connection is specifically as follows:

[0091]

[0092] Where, X∈R W+H+C It is the input tensor; f φl It is the feature extractor f φ The l-th layer (convolutional layer) has weights φl;

[0093] like Figure 3 As shown, A α It is a task-specific adapter, parameterized by parameter α. Matrix multiplication is specifically as follows:

[0094] A α (X) = X*α;

[0095] Where, α∈R C×C , * indicates a convolution operation (implemented using a 1x1 convolution kernel).

[0096] Specifically, such as Figure 4 As shown, after inputting the image feature tensor, it is first processed by a 3×3 convolutional layer and a 1×1 adapter branch, and then fused through residual connections ⊕. The fused features are then processed by batch normalization and activation functions, and this unit structure is repeated 16 times. Finally, after global average pooling, the output is a feature representation that can be used for subsequent classification and other tasks.

[0097] Step (10): Calculate S jThe corresponding rotated image input f φ,α The mean of the obtained features is used as the nearest centroid classifier (NCC), and then S j The corresponding rotated image input f φ,α The obtained features are fed into the nearest centroid classifier (NCC) and the feature measurement model, respectively. In support set S j Optimize Task-Specific Adapter A α The weight α is set, and the objective is to minimize the combined loss of its classification loss and rotation angle classification loss, i.e.:

[0098]

[0099] Among them, X rot It's S j Corresponding rotated image input; L ce It is the classification loss, calculated using NCC; L ssl The rotation angle classification loss is calculated using a feature measurement model; λ is the weight balancing the two losses; f φ,α It is a feature extractor that has been adapted by an adapter; the learning rate is dynamically adjusted by an Adadelta optimizer to optimize the weights λ.

[0100] Step (11): Use the optimized feature extractor f φ,α For query set Q j Feature extraction is performed and the data is sent to the NCC for classification; no feature measurement model is required. Calculate and record classification accuracy to test and evaluate the performance of the entire image recognition model on the current task.

[0101] Step (12): Repeat the above process, i.e. steps (7) to (11), for multiple small sample tasks in the target domain, calculate the average performance index on all tasks, and obtain the test results of the final model.

[0102] In summary, the cross-domain few-shot image recognition method trained during testing provided by this invention first performs rotation enhancement on the images in the support set and query set. Then, it uses a feature extractor f containing 3×3 convolutions and a 1×1 adapter to perform batch normalization (BN), ReLU activation function, and global average pooling (GAP) to obtain rotated image features. On the other hand, it utilizes a self-supervised learning model g... θ Predict the image rotation angle to minimize the rotation angle classification loss L ssl On the one hand, semantically stable features are learned, and on the other hand, the nearest centroid classifier (NCC) is used for class prediction, with a classification loss L. ceBy ensuring classification discriminativeness, employing dual-loss synergy, and relying on a lightweight adapter to adapt to the target domain, the trained image recognition model can achieve accurate classification in cross-domain, low-sample scenarios, unlocking the value of low-sample data and balancing general semantics with classification requirements.

[0103] Based on the cross-domain few-shot image recognition method trained during the above tests, the present invention provides an embodiment of the corresponding system item.

[0104] See Figure 5 , Figure 5 This is a structural block diagram of a test-time trained cross-domain few-shot image recognition system provided in an embodiment of the present invention. The test-time trained cross-domain few-shot image recognition system 20 includes:

[0105] The self-supervised learning model training module 21 is used to extract rotation image features of the source domain training set images using the first feature extractor, train the self-supervised learning model, and obtain the optimized self-supervised learning model.

[0106] The feature extractor training module 22 is used to freeze the parameters of the first feature extractor and the optimized self-supervised learning model, and to extract rotational image features of the target domain support set image using the second feature extractor as input to the optimized self-supervised learning model to obtain a rotation angle classification loss; to calculate the mean of each type of rotational image feature as the nearest centroid classifier, and to use the rotational image features of the target domain support set image as input to the nearest centroid classifier to obtain a classification loss; and to optimize the parameters of the second feature extractor by minimizing the combined loss to obtain the optimized second feature extractor and the trained image recognition model; wherein, the combined loss includes the classification loss and the rotation angle classification loss;

[0107] The image recognition test module 23 is used to extract features of the target domain query set image using the optimized second feature extractor as input to the nearest centroid classifier, and to perform recognition tests on the image recognition model based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

[0108] It should be noted that the test-time trained cross-domain few-shot image recognition system provided in this embodiment of the invention is used to execute all the process steps of the test-time trained cross-domain few-shot image recognition method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0109] This invention also provides a terminal device, such as... Figure 6The diagram shown is a structural block diagram of a preferred embodiment of a terminal device provided by the present invention. The terminal device includes a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31. When the processor 31 executes the computer program, it implements the cross-domain few-shot image recognition method trained during testing as described in any of the above embodiments.

[0110] In addition, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the cross-domain few-sample image recognition method trained during testing as described in any of the above embodiments.

[0111] When the processor 31 executes the computer program, it implements the steps in the above-described embodiments of the cross-domain few-shot image recognition method, for example... Figure 1 The steps of the cross-domain few-shot image recognition method trained during test-time are shown. Alternatively, when the processor 31 executes the computer program, it implements the functions of each module in the above-described embodiment of the cross-domain few-shot image recognition system trained during test-time, for example... Figure 5 The diagram shows the functionality of each module in the cross-domain few-shot image recognition system trained during testing.

[0112] Preferably, the computer program can be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.

[0113] The processor 31 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 31 can be any conventional processor. The processor 31 is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.

[0114] The memory 32 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 32 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), and a flash card, etc., or the memory 32 can also be other volatile solid-state storage devices.

[0115] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 6 The structural block diagram shown is merely a structural example of the terminal device described above and does not constitute a limitation on the structure of the terminal device. The terminal device may include more or fewer components than shown, or combine certain components, or use different components.

[0116] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A cross-domain few-shot image recognition method trained during testing, characterized in that, include: The first feature extractor is used to extract rotational image features from the source domain training set images, and the self-supervised learning model is trained to obtain the optimized self-supervised learning model. The parameters of the first feature extractor and the optimized self-supervised learning model are frozen. The second feature extractor extracts rotational image features from the target domain support set image as input to the optimized self-supervised learning model, resulting in a rotation angle classification loss. The mean of each type of rotational image feature is calculated as the nearest centroid classifier, and the rotational image features from the target domain support set image are used as input to the nearest centroid classifier, resulting in a classification loss. The parameters of the second feature extractor are optimized by minimizing the combined loss, resulting in an optimized second feature extractor and a trained image recognition model. The combined loss includes the classification loss and the rotation angle classification loss. The optimized second feature extractor is used to extract features from the target domain query set image, which are then used as input to the nearest centroid classifier. The image recognition model is then tested based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

2. The cross-domain few-shot image recognition method trained during testing as described in claim 1, characterized in that, The second feature extractor is obtained by connecting the residuals of the first feature extractor to a task-specific adapter, which is parameterized by specific parameters. The residual connection method is as follows: Where X is the input tensor, f φl f represents the first feature extractor φ The l-th layer has a weight of φl and A α A represents the task-specific adapter obtained by parameterizing parameter α. α (X) = X*α, where * represents the convolution operation.

3. The cross-domain few-shot image recognition method trained during testing as described in claim 1, characterized in that, The process involves using a first feature extractor to extract rotational image features from the source domain training set images, training the self-supervised learning model, and obtaining an optimized self-supervised learning model, including: The first feature extractor is used to extract rotational image features from the source domain training set images; The rotation image features of the source domain training set images are used as input to the self-supervised learning model. The parameters of the first feature extractor are frozen, and the parameters of the self-supervised learning model are optimized by minimizing the rotation angle classification loss, resulting in an optimized self-supervised learning model.

4. The cross-domain few-shot image recognition method trained during testing as described in claim 3, characterized in that, The step of extracting rotational image features from the source domain training set images using a first feature extractor includes: The source domain training set images are preprocessed by rotation operators to generate rotated images with four rotation angles, thus obtaining rotated images of the source domain training set images; wherein the rotation angles include 0°, 90°, 180° and 270°. The first feature extractor is used to extract features from the rotated images of the source domain training set images to obtain the rotated image features of the source domain training set images.

5. The cross-domain few-shot image recognition method trained during testing as described in claim 4, characterized in that, The rotation angle classification loss is: in, g is the total number of samples in the source domain training set images. θ f represents a self-supervised learning model. φ This represents the first feature extractor. Represents the input image X in the training set. m The rotated image at the k-th rotation angle, This represents the predicted probability for the k-th rotation angle.

6. The cross-domain few-shot image recognition method trained during testing as described in claim 3, characterized in that, After taking the rotation image features of the source domain training set images as input to the self-supervised learning model, freezing the parameters of the first feature extractor, and optimizing the parameters of the self-supervised learning model by minimizing the rotation angle classification loss to obtain the optimized self-supervised learning model, the method further includes: The rotation image features extracted from the source domain validation set images by the first feature extractor are used as input to the optimized self-supervised learning model. The hyperparameters of the optimized self-supervised learning model are then optimized based on the accuracy of the optimized self-supervised learning model until the hyperparameters of the optimized self-supervised learning model meet the preset hyperparameter conditions of the self-supervised learning model, thus obtaining the final self-supervised learning model.

7. The cross-domain few-shot image recognition method trained during testing as described in claim 5, characterized in that, The objective function for minimizing the combination loss is: Where α represents the task-specific adapter A α The parameter N; S S represents the total number of samples in the target domain support set of images; j X represents the target domain support set; rot S represents j The corresponding rotated image input; y represents the category label corresponding to the image in the target domain support set; y ssl The L represents the rotation angle category label corresponding to the rotated image of the target domain support set image; ce The classification loss is represented by L, calculated using the nearest centroid classifier. ssl λ is the rotation angle classification loss, calculated using the optimized self-supervised learning model; f is the weight balancing the two losses; φ,α This refers to the second feature extractor.

8. The cross-domain few-shot image recognition method trained during testing as described in claim 1, characterized in that, The first feature extractor is obtained by supervised learning on a large-scale dataset.

9. A cross-domain few-shot image recognition system trained during testing, characterized in that, include: The self-supervised learning model training module is used to extract rotation image features from the source domain training set images using the first feature extractor, train the self-supervised learning model, and obtain the optimized self-supervised learning model. The feature extractor training module is used to freeze the parameters of the first feature extractor and the optimized self-supervised learning model. It uses a second feature extractor to extract rotational image features from the target domain support set image as input to the optimized self-supervised learning model, obtaining a rotation angle classification loss. It calculates the mean of each type of rotational image feature as the nearest centroid classifier, using the rotational image features from the target domain support set image as input to the nearest centroid classifier, obtaining a classification loss. It optimizes the parameters of the second feature extractor by minimizing the combined loss, obtaining the optimized second feature extractor and the trained image recognition model. The combined loss includes the classification loss and the rotation angle classification loss. The image recognition testing module is used to extract features from the target domain query set image using the optimized second feature extractor as input to the nearest centroid classifier, and to perform recognition testing on the image recognition model based on the accuracy of the nearest centroid classifier to obtain the image recognition result.

10. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the test-time trained cross-domain few-shot image recognition method as described in any one of claims 1 to 8.

Images