Small sample open set image recognition method based on low-dimensional contrast adaptation

Abstract

The present invention discloses a small sample open set image recognition method based on low-dimensional contrast adaptation. The method of the present invention first samples the open image data set to obtain a support set and a query set, inputs them into a visual feature extraction module, and obtains a corresponding feature vector set; then, a low-dimensional contrast embedding module is used to convert the feature vector set into a low-dimensional dense feature vector set; then a task adaptive correction module is constructed, and the support feature vector set is corrected and reconstructed by a low-dimensional dense support feature vector set to obtain a support prototype vector set; finally, the low-dimensional dense feature vector set, the support prototype vector set, and the query feature vector set are input into a metric classification module to obtain the sample category in the query set. The method of the present invention uses a low-dimensional contrast embedding module to dynamically mine the representation mode of unknown class samples, thereby improving the recognition accuracy of unknown classes. At the same time, the task adaptive correction module is used to reduce the interference of similar visual features in unknown categories, thereby improving the classification accuracy of known categories.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, in particular to the field of image recognition in a small sample open set scenario, and relates to a small sample open set image recognition method based on low-dimensional contrast adaptation. Background Art

[0002] Open set image recognition is an extension of the general image recognition task, which aims to correctly classify images of known categories defined in the training set and identify images of unknown categories. In recent years, deep learning models based on deep convolutional networks have achieved excellent performance in open set image recognition tasks in scenarios with rich data samples. However, traditional methods cannot be directly applied to scenarios with relatively scarce data samples such as rare animal recognition. The learned model may overfit when the training data is insufficient, resulting in weak generalization ability on new data samples. To this end, from the perspective of human cognition of new things, researchers have proposed the theory of small sample learning and explored methods to solve the problem of small sample open set image recognition. Small sample open set image recognition aims to identify unlabeled query set samples through a small number of support set samples with category labels, where the categories of the support set samples are known, and the categories in these open sets may not appear in the training set and the query set samples can be any unknown category, that is, the category label space corresponding to the support set and the query set is inconsistent. The small sample open set image recognition model needs to distinguish the query set samples of known categories and identify the query set samples of unknown categories. It can be applied to open scenarios with relatively scarce data such as scarce drug discovery and military target recognition. For example, in the field of drug discovery, there are few drug samples with the same molecular structure, and there are unknown new structural drugs. Existing methods cannot simultaneously solve the problems of data scarcity and open categories in this task, which is not conducive to accurate decision-making by professionals.

[0003] Most of the existing research on small sample open set recognition has evolved from conventional small sample and open set recognition problems, and adopts the framework of metric learning. In order to simulate real application scenarios, these methods all train models according to the meta-learning paradigm. In the training phase, the training set is divided into "tasks" (tasks or episodes). The data sets in different "tasks" contain their own support sets and query sets. The query set contains known support set categories and other unknown categories. In each iterative optimization, the support set is used as a set of labeled samples in the "task". The model needs to use a small number of support samples to correctly classify known class samples in the query set and identify unknown class samples, and then optimize according to the classification loss of the query set samples. Generally, the typical method inputs the support set and query set into the feature embedding network to obtain the corresponding embedding vector, and then inputs the embedding vector into the metric module for similarity comparison and classification. For query samples of unknown categories, the model usually pre-sets a threshold. If the maximum known class score corresponding to the query sample is less than this threshold, the query sample is regarded as an unknown category.

[0004] The above-mentioned small sample open set recognition methods have the following main shortcomings: (1) The known categories will change with the support set of different "tasks", and the similarity between categories is inconsistent. The fixed threshold cannot dynamically and adaptively divide the decision boundaries of known and unknown categories in different "tasks"; (2) The samples in the query set can come from any unknown category, and their specific distribution cannot be estimated, which makes it difficult for the model to mine effective unknown class representation information from a small number of support samples; (3) The query set may contain unknown categories with the same visual features as known categories, which can easily cause further interference to the classification of known categories. In summary, in order to alleviate the problem of inaccurate recognition caused by the inability of existing methods to dynamically divide decision boundaries and reasonably represent unknown categories, there is an urgent need for a method that does not require a fixed threshold, can mine unknown class representation information based on a small number of known category samples in the support set, and reduce the interference of similar visual features. Summary of the invention

[0005] The purpose of the present invention is to provide a small sample open set image recognition method based on low-dimensional contrast adaptation in view of the deficiencies of the prior art. The method of the present invention designs a low-dimensional contrast embedding module, dynamically mines the low-dimensional dense representation of the unknown class based on the known class, and solves the problem of difficult decision boundary division caused by the change of the known class; uses the task adaptive correction module to update the original known class features, solves the interference problem of similar visual features in the unknown class, and finally accurately distinguishes the known class and identifies the unknown class.

[0006] The method of the present invention first obtains an open image dataset consisting of samples with known category annotations and some samples with unknown category, and then performs the following operations:

[0007] Step (1) sampling an open image dataset to obtain a support set and a query set containing some unknown category samples, inputting both into a visual feature extraction module, and outputting a support feature vector set and a query feature vector set;

[0008] Step (2) constructing a low-dimensional contrast embedding module, the input of which is a support feature vector set and a query feature vector set, and the output of which is a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set;

[0009] Step (3) constructing a task adaptive correction module, the input of which is a support feature vector set and a low-dimensional dense support feature vector set, and the output of which is a support prototype vector set;

[0010] Step (4) constructing a metric classification module, inputting a low-dimensional dense support feature vector set, a low-dimensional dense query feature vector set, a support prototype vector set and a query feature vector set, and outputting a predicted category label of a sample in the query set;

[0011] Step (5) uses the stochastic gradient descent algorithm to optimize the small sample open set image recognition model composed of a low-dimensional contrast embedding module, a task adaptive correction module and a metric classification module, and obtains the image sample categories in the query set through steps (1) to (4) for the new support set and query set.

[0012] Further, step (1) specifically comprises:

[0013] (1-1) All images in the open image dataset are scaled to the same size, and the scaled image dataset is randomly sampled without replacement to obtain a known class label set As well as the support set and query set, C represents the total number of sample categories in the support set; the support set The superscript s indicates support, N s represents the number of image samples in the support set and N s =C×K, K represents the number of samples of each category in the support set, Represents the i-th support image sample and the corresponding category label; query set The superscript q represents query, N q represents the number of image samples in the query set, represents the j-th query image sample and the corresponding category label, and U represents the unknown class label set, indicating that there are samples from unknown categories in the query set; represents the real number domain, H represents the image height, W represents the image width, and 3 represents the number of RGB channels;

[0014] (1-2) Construct a visual feature extraction module consisting of a deep convolutional network and a fully connected layer, where the deep convolutional network is an 18-layer residual network ResNet-18 (Residual Network-18) pre-trained on the large-scale image dataset ImageNet, and the output dimension of the fully connected layer is d;

[0015] (1-3) will support the set and queryset Input to the visual feature extraction module to obtain the support feature vector set and query feature vector set in represents the i-th supporting eigenvector, represents the jth query feature vector, and d represents the dimension of a single feature vector.

[0016] Furthermore, step (2) is specifically:

[0017] (2-1) Construct a low-dimensional contrast embedding module, which consists of a low-dimensional projection unit, a reconstruction error auxiliary loss, and a supervised contrast auxiliary loss. The low-dimensional projection unit projects the sample into a binary low-dimensional space, and the reconstruction error auxiliary loss retains the key visual features of the sample as much as possible. The supervised contrast auxiliary loss increases the distance between known class samples and unknown class samples in the low-dimensional space, and reduces the distance between the low-dimensional features of samples between known classes, thereby mining more reasonable visual features of unknown classes.

[0018] (2-2) will support the feature vector set and the query feature vector Input to the low-dimensional projection unit to obtain a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set The i-th low-dimensional dense support eigenvector The j-th low-dimensional dense query feature vector d′ represents the dimension of the low-dimensional vector after projection and d′<<d, represents the low-dimensional projection parameter matrix, T represents the matrix transpose operation;

[0019] (2-3) According to each low-dimensional dense support vector and a low-dimensional dense support query vector Calculate the low-dimensional dense feature reconstruction auxiliary loss loss1 to retain the key visual features of the original high-dimensional space as much as possible. Represents a vector The square of the norm, λ represents the regularization term The scaling factor hyperparameter is used to suppress overfitting, λ>0;

[0020] (2-4) According to the low-dimensional dense vector and the query set class label of the corresponding sample Calculate the supervised contrast auxiliary loss loss2, exp(·) represents the exponential function with the natural constant e as the base, cos(·,·) represents the cosine similarity function, τ represents the similarity scaling factor hyperparameter, τ>0, and They represent query samples whose corresponding classes are labeled as known classes and unknown classes, respectively.

[0021] Furthermore, step (3) is specifically:

[0022] (3-1) Constructing a task adaptive correction module. The task adaptive correction module uses the correlation between inter-class vectors to construct an inter-class relationship vector set, adaptively adjusts the feature representation of known classes, and makes the features of different known classes more distinguishable. At the same time, the class features are corrected and reconstructed through a low-dimensional dense support feature vector set to remove irrelevant information of unknown classes and reduce the interference of similar visual features in unknown classes on known classes.

[0023] (3-2) According to the supporting eigenvector Get the initial support prototype vector set Support prototype vector for category c K represents the number of samples of each category in the support set. Indicates that all samples of category c in the support set;

[0024] (3-3) Using the Support Prototype Vector Set Construct the feature relationship between known classes in known tasks and obtain the inter-class relationship vector set Relation vector of category c FFN(·) represents a feedforward neural network consisting of two fully connected layers, c′ represents other known categories except c, and |·| represents taking the absolute value;

[0025] (3-4) Based on the low-dimensional dense support vector set Get the inter-class correction matrix Used to correct the support prototype vector set The value of the cth row and ith column of matrix W represents the parameter matrix used to align the vector dimensions, dist(·,·) represents the Euclidean distance function, represents the support prototype vector of category κ;

[0026] (3-5) Using the inter-class correction matrix W and the low-dimensional dense support vector set and the inter-class relationship vector set Original support prototype vector set Perform correction and reconstruction to obtain the corrected prototype vector set The specific formula for the correction prototype vector of category c is: ⊙ represents an element-by-element multiplication operation.

[0027] Furthermore, step (4) is specifically:

[0028] (4-1) Construct a metric classification module, which consists of a binary classifier for unknown classes, a multi-classifier for known classes, and a classification loss;

[0029] (4-2) Based on the low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set Get the correlation matrix The value of the jth row and ith column of the matrix M <·,·> represents the dot product operation of vectors;

[0030] (4-3) Obtain the unknown class binary probability matrix based on the correlation matrix M Ο′=σ(MLP(M)), where MLP(·) represents a multilayer perceptron consisting of three linear hidden layers and 2 output units, σ(·) represents the Sigmoid function that scales the output value to the probability value interval (0,1), and the two-dimensional vector o′ in the j-th row of the matrix Ο′ is j =(o′ j,0 ,o′ j,1 ) in the 0th column value o′ j,0 and the first column value o′ j,1 Represent query samples respectively The probability of belonging to an unknown class and the probability of belonging to a known class;

[0031] (4-4) According to the correction prototype vector set obtained in step (3-5) And the query feature vector set obtained in steps (1-3) Get the known class probability matrix The value of the jth row and cth column of the matrix O″ o″ j,c Represents a query sample The probability of belonging to the cth class, ω is the scaling factor hyperparameter, which adjusts the calculated distance value, ω>0, represents the corrected prototype vector of category κ;

[0032] (4-5) Using the unknown class binary probability matrix O′ and the known class probability matrix O″ to obtain the query set Each query sample The predicted class label of (o″ j,1 ,o″j,2 ,…,o″ j,C ) represents the probability vector of the j-th row of the matrix O″, Representation sample The predicted category of is unknown, argmax(·) represents the operation of taking the independent variable that maximizes the function value, and the independent variable in the above formula is category c;

[0033] (4-6) Through each query sample The true category label The unknown class binary probability matrix O′ and the known class probability matrix O″ are used to calculate the query set The classification loss loss3, Among them, a single query sample The classification loss express Belongs to the unknown category.

[0034] Furthermore, step (5) is specifically:

[0035] (5-1) According to the low-dimensional dense feature reconstruction auxiliary loss loss1, the supervised contrast auxiliary loss loss2 and the classification loss loss3, the total model loss loss = loss1 + loss2 + loss3 is obtained, and the small sample open set image recognition model composed of the low-dimensional contrast embedding module, the task adaptive correction module and the metric classification module is optimized by the stochastic gradient descent algorithm. The model is iteratively trained until convergence to obtain the optimized small sample open set recognition model;

[0036] (5-2) Sampling the new open image set to obtain the support set and an unlabeled query set with unknown categories Input the optimized small sample open set model and execute steps (1) to (4) in sequence to output the query set All samples in The predicted class label of in represents the number of image samples in the support set, Indicates support image samples and corresponding category labels, represents a known set of categories, is the total number of known categories, represents the number of image samples in the query set, Indicates query image samples, For query samples Predict class label, if but show is predicted to be of unknown class.

[0037] The present invention proposes a small sample open set image recognition method based on low-dimensional contrast adaptation, which has the following characteristics: (1) A low-dimensional contrast embedding module is used to transform the unknown class recognition into a binary classification problem without the need for a predefined fixed threshold, thereby improving the recognition accuracy of the unknown class; (2) According to the correlation between the known class vectors in the support set, an inter-class relationship feature vector is adaptively constructed to make the features of different known classes more distinguishable, thereby improving the classification accuracy of the known classes; (3) The learned low-dimensional dense features are fully utilized to correct the features of the known classes, thereby reducing the interference of similar features in the unknown class on the classification.

[0038] The present invention is applicable to image recognition tasks in small sample open set scenarios, and has the following beneficial effects: (1) the samples are projected into low-dimensional dense space through a low-dimensional contrast embedding module, and reasonable unknown class sample representations are dynamically mined according to known classes, without the need for fixed thresholds for decision making, thereby improving the recognition accuracy of unknown classes; (2) the known class features are adaptively adjusted according to the correlation between known class vectors, making different known classes more distinguishable, thereby improving the classification accuracy of known classes; (3) the task adaptive correction module uses the mined low-dimensional dense discriminant feature information to correct and reconstruct the known class features, suppressing visual features similar to the unknown class, thereby reducing the classification interference of the unknown class on the known class. The low-dimensional contrast embedding module and the adaptive correction module of the present invention significantly improve the performance of the small sample open set image recognition model, and can be applied to practical application fields such as drug discovery and military target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings.

[0041] like Figure 1, a small sample open set image recognition method based on low-dimensional contrast adaptation is firstly used to sample the open image dataset to obtain the support set and query set, and then use the visual feature extraction module to obtain the support feature vector set and the query feature vector set; then the support feature vector set and the query feature vector set are input into the low-dimensional contrast embedding module to obtain the low-dimensional dense support feature vector set and the low-dimensional dense query feature vector set; then the task adaptive correction module is constructed, and the support feature vector set is corrected and reconstructed by the low-dimensional dense support feature vector set to obtain the support prototype vector set; finally, the low-dimensional dense support feature vector set, the low-dimensional dense query feature vector set, the support prototype vector set and the query feature vector set are input into the metric classification module to obtain the sample categories in the query set. This method uses the low-dimensional contrast embedding module to dynamically mine reasonable unknown class sample representations without the need for fixed thresholds for decision making, thereby improving the recognition accuracy of unknown classes. At the same time, the task adaptive correction module is used to correct and reconstruct the known class features, reduce the interference of similar features in the unknown class, and thus improve the classification accuracy of the known class.

[0042] This method first obtains an open image dataset consisting of samples with known category annotations and some samples with unknown category annotations, and then performs the following operations:

[0043] Step (1) samples the open image dataset to obtain a support set and a query set containing some unknown category samples, inputs the two into a visual feature extraction module, and outputs a support feature vector set and a query feature vector set; specifically:

[0044] (1-1) All images in the open image dataset are scaled to the same size, and the scaled image dataset is randomly sampled without replacement to obtain a known class label set As well as the support set and query set, C represents the total number of sample categories in the support set; the support set The superscript s indicates support, N s represents the number of image samples in the support set and N s =C×K, K represents the number of samples of each category in the support set, Represents the i-th support image sample and the corresponding category label; query set The superscript q represents query, N q represents the number of image samples in the query set, represents the j-th query image sample and the corresponding category label, Represents the unknown class label set, indicating that there are samples from unknown categories in the query set; represents the real number domain, H represents the image height, W represents the image width, and 3 represents the number of RGB channels;

[0045] (1-2) Construct a visual feature extraction module consisting of a deep convolutional network and a fully connected layer, where the deep convolutional network is an 18-layer residual network ResNet-18 (Residual Network-18) pre-trained on the large-scale image dataset ImageNet, and the output dimension of the fully connected layer is d;

[0046] (1-3) will support the set and queryset Input to the visual feature extraction module to obtain the support feature vector set and query feature vector set in represents the i-th supporting eigenvector, represents the jth query feature vector, and d represents the dimension of a single feature vector.

[0047] Step (2) constructs a low-dimensional contrast embedding module, the input of which is a support feature vector set and a query feature vector set, and the output is a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set; specifically:

[0048] (2-1) Construct a low-dimensional contrast embedding module, which consists of a low-dimensional projection unit, a reconstruction error auxiliary loss, and a supervised contrast auxiliary loss. The low-dimensional projection unit projects the sample into a binary low-dimensional space, and the reconstruction error auxiliary loss retains the key visual features of the sample as much as possible. The supervised contrast auxiliary loss increases the distance between known class samples and unknown class samples in the low-dimensional space, and reduces the distance between the low-dimensional features of samples between known classes, thereby mining more reasonable visual features of unknown classes.

[0049] (2-2) will support the feature vector set and the query feature vector Input to the low-dimensional projection unit to obtain a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set The i-th low-dimensional dense support eigenvector The j-th low-dimensional dense query feature vector d′ represents the dimension of the low-dimensional vector after projection and d′<<d, represents the low-dimensional projection parameter matrix, T represents the matrix transpose operation;

[0050] (2-3) According to each low-dimensional dense support vector and a low-dimensional dense support query vector Calculate the low-dimensional dense feature reconstruction auxiliary loss loss1 to retain the key visual features of the original high-dimensional space as much as possible. Represents a vector The square of the norm, λ represents the regularization term The scaling factor hyperparameter is used to suppress overfitting, λ>0, and λ=0.2 in this embodiment;

[0051] (2-4) According to the low-dimensional dense vector and the query set class label of the corresponding sample Calculate the supervised contrast auxiliary loss loss2, exp(·) represents an exponential function with the natural constant e as the base, cos(·,·) represents the cosine similarity function, τ represents a similarity scaling factor hyperparameter, τ>0, in this embodiment τ=0.5, and They represent query samples whose corresponding classes are labeled as known classes and unknown classes, respectively.

[0052] Step (3) constructs a task adaptive correction module, the input is a support feature vector set and a low-dimensional dense support feature vector set, and the output is a support prototype vector set; specifically:

[0053] (3-1) Constructing a task adaptive correction module. The task adaptive correction module uses the correlation between inter-class vectors to construct an inter-class relationship vector set, adaptively adjusts the feature representation of known classes, and makes the features of different known classes more distinguishable. At the same time, the class features are corrected and reconstructed through a low-dimensional dense support feature vector set to remove irrelevant information of unknown classes and reduce the interference of similar visual features in unknown classes on known classes.

[0054] (3-2) According to the supporting eigenvector Get the initial support prototype vector set Support prototype vector for category c K represents the number of samples of each category in the support set. Indicates that all samples of category c in the support set;

[0055] (3-3) Using the Support Prototype Vector Set Construct the feature relationship between known classes in known tasks and obtain the inter-class relationship vector set Relation vector of category c FFN(·) represents a feedforward neural network consisting of two fully connected layers, c′ represents other known categories except c, and |·| represents taking the absolute value;

[0056] (3-4) Based on the low-dimensional dense support vector set Get the inter-class correction matrix Used to correct the support prototype vector set The value of the cth row and ith column of matrix W represents the parameter matrix used to align the vector dimensions, dist(·,·) represents the Euclidean distance function, represents the support prototype vector of category κ;

[0057] (3-5) Using the inter-class correction matrix W and the low-dimensional dense support vector set and the inter-class relationship vector set Z to the original support prototype vector set Perform correction and reconstruction to obtain the corrected prototype vector set The specific formula for the correction prototype vector of category c is: ⊙ represents an element-by-element multiplication operation.

[0058] Step (4) constructs a metric classification module, inputs a low-dimensional dense support feature vector set, a low-dimensional dense query feature vector set, a support prototype vector set and a query feature vector set, and outputs the predicted category label of the sample in the query set; specifically:

[0059] (4-1) Construct a metric classification module, which consists of a binary classifier for unknown classes, a multi-classifier for known classes, and a classification loss;

[0060] (4-2) Based on the low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set Get the correlation matrix The value of the jth row and ith column of the matrix M <·,·> represents the dot product operation of vectors;

[0061] (4-3) Obtain the unknown class binary probability matrix based on the correlation matrix M Ο′=σ(MLP(M)), where MLP(·) represents a multilayer perceptron consisting of three linear hidden layers and 2 output units, σ(·) represents the Sigmoid function that scales the output value to the probability value interval (0,1), and the two-dimensional vector o′ in the j-th row of the matrix Ο′ is j =(o′ j,0 ,o′ j,1 ) in the 0th column value o′ j,0 and the first column value o′ j,1 Represent query samples respectively The probability of belonging to an unknown class and the probability of belonging to a known class;

[0062] (4-4) According to the correction prototype vector set obtained in step (3-5) And the query feature vector set obtained in steps (1-3) Get the known class probability matrix The value of the jth row and cth column of the matrix O″ o″ j,c Represents a query sample The probability of belonging to the cth class, ω is a scaling factor hyperparameter, which adjusts the calculated distance value, ω>0, in this embodiment ω=2, represents the corrected prototype vector of category κ;

[0063] (4-5) Using the unknown class binary probability matrix O′ and the known class probability matrix O″ to obtain the query set Each query sample The predicted class label of (o″ j,1 ,o″ j,2 ,…,o″ j,C ) represents the probability vector of the j-th row of the matrix O″, Representation sample The predicted category of is unknown, argmax(·) represents the operation of taking the independent variable that maximizes the function value, and the independent variable in the above formula is category c;

[0064] (4-6) Through each query sample The true category label The unknown class binary probability matrix O′ and the known class probability matrix O″ calculate the query set The classification loss loss3, Among them, a single query sample The classification loss express Belongs to the unknown category.

[0065] Step (5) uses a stochastic gradient descent algorithm to optimize a small sample open set image recognition model consisting of a low-dimensional contrast embedding module, a task adaptive correction module, and a metric classification module, and obtains the image sample categories in the query set through steps (1) to (4) for the new support set and query set; specifically:

[0066] (5-1) According to the low-dimensional dense feature reconstruction auxiliary loss loss1, the supervised contrast auxiliary loss loss2 and the classification loss loss3, the total model loss loss = loss1 + loss2 + loss3 is obtained, and the small sample open set image recognition model composed of the low-dimensional contrast embedding module, the task adaptive correction module and the metric classification module is optimized by the stochastic gradient descent algorithm. The model is iteratively trained until convergence to obtain the optimized small sample open set recognition model;

[0067] (5-2) Sampling the new open image set to obtain the support set and an unlabeled query set with unknown categories Input the optimized small sample open set model and execute steps (1) to (4) in sequence to output the query set All samples in The predicted class label of in represents the number of image samples in the support set, Indicates support image samples and corresponding category labels, represents a known set of categories, is the total number of known categories, represents the number of image samples in the query set, Indicates query image samples, For query samples Predict class label, if but show is predicted to be of unknown class.

[0068] The contents described in this embodiment are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as being limited to the specific forms described in the embodiment. The protection scope of the present invention also extends to the equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A small sample open set image recognition method based on low-dimensional contrast adaptation, characterized in that: First, obtain an open image dataset consisting of samples with known category labels and some samples with unknown category labels, and then perform the following operations: Step (1) sampling an open image dataset to obtain a support set and a query set containing some unknown category samples, inputting both into a visual feature extraction module, and outputting a support feature vector set and a query feature vector set; Specifically: (1-1) All images in the open image dataset are scaled to the same size, and the scaled image dataset is randomly sampled without replacement to obtain a known class label set As well as the support set and query set, C represents the total number of sample categories in the support set; the support set The superscript s indicates support, N s represents the number of image samples in the support set and N s =C×K, K represents the number of samples of each category in the support set, Represents the i-th support image sample and the corresponding category label; query set The superscript q indicates a query, N q represents the number of image samples in the query set, represents the j-th query image sample and the corresponding category label, Represents the unknown class label set, indicating that there are samples from unknown categories in the query set; represents the real number domain, H represents the image height, W represents the image width, and 3 represents the number of RGB channels; (1-2) Construct a visual feature extraction module consisting of a deep convolutional network and a fully connected layer, where the deep convolutional network is an 18-layer residual network ResNet-18 pre-trained on the large-scale image dataset ImageNet, and the output dimension of the fully connected layer is d; (1-3) will support the set and queryset Input to the visual feature extraction module to obtain the support feature vector set and query feature vector set in represents the i-th supporting eigenvector, represents the jth query feature vector, and d represents the dimension of a single feature vector; Step (2) constructs a low-dimensional contrast embedding module, the input of which is a support feature vector set and a query feature vector set, and the output is a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set; specifically: (2-1) Construct a low-dimensional contrast embedding module, which consists of a low-dimensional projection unit, a reconstruction error auxiliary loss, and a supervised contrast auxiliary loss. The low-dimensional projection unit projects the sample into a binary low-dimensional space, and the reconstruction error auxiliary loss retains the key visual features of the sample as much as possible. The supervised contrast auxiliary loss increases the distance between known class samples and unknown class samples in the low-dimensional space, and reduces the distance between the low-dimensional features of samples between known classes, thereby mining more reasonable visual features of unknown classes. (2-2) will support the feature vector set and the query feature vector Input to the low-dimensional projection unit to obtain a low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set The i-th low-dimensional dense support eigenvector The j-th low-dimensional dense query feature vector d′ represents the dimension of the low-dimensional vector after projection and d′<<d, represents the low-dimensional projection parameter matrix, T represents the matrix transpose operation; (2-3) According to each low-dimensional dense support vector and a low-dimensional dense support query vector Calculate the low-dimensional dense feature reconstruction auxiliary loss loss1 to retain the key visual features of the original high-dimensional space as much as possible. represents the square of the l2 norm of the vector, and λ represents the regularization term The scaling factor hyperparameter is used to suppress overfitting, λ>0; (2-4) According to the low-dimensional dense vector and the query set class label of the corresponding sample Compute supervised contrastive auxiliary loss exp(·) represents the exponential function with the natural constant e as the base, cos(·,·) represents the cosine similarity function, τ represents the similarity scaling factor hyperparameter, τ>0, and Respectively represent query samples whose corresponding classes are marked as known classes and unknown classes; Step (3) constructs a task adaptive correction module, the input is a support feature vector set and a low-dimensional dense support feature vector set, and the output is a support prototype vector set; specifically: (3-1) Constructing a task adaptive correction module. The task adaptive correction module uses the correlation between inter-class vectors to construct an inter-class relationship vector set, adaptively adjusts the feature representation of known classes, and makes the features of different known classes more distinguishable. At the same time, the class features are corrected and reconstructed through a low-dimensional dense support feature vector set to remove irrelevant information of unknown classes and reduce the interference of similar visual features in unknown classes on known classes. (3-2) According to the supporting eigenvector Get the initial support prototype vector set Support prototype vector for category c K represents the number of samples of each category in the support set. Indicates that all samples of category c in the support set; (3-3) Using the Support Prototype Vector Set Construct the feature relationship between known classes in known tasks and obtain the inter-class relationship vector set Relation vector of category c FFN(·) represents a feedforward neural network consisting of two fully connected layers, c′ represents other known categories except c, and |·| represents taking the absolute value; (3-4) Based on the low-dimensional dense support vector set Get the inter-class correction matrix Used to correct the support prototype vector set The value of the cth row and ith column of matrix W represents the parameter matrix used to align the vector dimensions, dist(·,·) represents the Euclidean distance function, represents the support prototype vector of category κ; (3-5) Using the inter-class correction matrix W and the low-dimensional dense support vector set and the inter-class relationship vector set Original support prototype vector set Perform correction and reconstruction to obtain the corrected prototype vector set The specific formula for the correction prototype vector of category c is: ⊙ represents element-by-element multiplication operation; Step (4) constructing a metric classification module, inputting a low-dimensional dense support feature vector set, a low-dimensional dense query feature vector set, a support prototype vector set and a query feature vector set, and outputting a predicted category label of a sample in the query set; Step (5) uses the stochastic gradient descent algorithm to optimize the small sample open set image recognition model composed of a low-dimensional contrast embedding module, a task adaptive correction module and a metric classification module, and obtains the image sample categories in the query set through steps (1) to (4) for the new support set and query set.

2. The small sample open set image recognition method based on low-dimensional contrast adaptation as claimed in claim 1, characterized in that: Step (4) is specifically: (4-1) Construct a metric classification module, which consists of a binary classifier for unknown classes, a multi-classifier for known classes, and a classification loss; (4-2) Based on the low-dimensional dense support feature vector set and a low-dimensional dense query feature vector set Get the correlation matrix The value of the jth row and ith column of matrix M <·,·> represents the dot product operation of vectors; (4-3) Obtain the unknown class binary probability matrix based on the correlation matrix M Ο′=σ(MLP(M)), where MLP(·) represents a multilayer perceptron consisting of three linear hidden layers and 2 output units, σ(·) represents the Sigmoid function that scales the output value to the probability value interval (0,1), and the two-dimensional vector o′ in the j-th row of the matrix Ο′ is j =(o′ j,0 ,o′ j,1 ) in the 0th column value o′ j,0 and the first column value o j ' ,1 Represent query samples respectively The probability of belonging to an unknown class and the probability of belonging to a known class; (4-4) According to the correction prototype vector set obtained in step (3-5) And the query feature vector set X obtained in step (1-3) q Get the known class probability matrix The value of the jth row and cth column of the matrix O″ o j ″ ,c Represents a query sample The probability of belonging to the cth class, ω is the scaling factor hyperparameter, which adjusts the calculated distance value, ω>0, represents the corrected prototype vector of category κ; (4-5) Using the unknown class binary probability matrix O′ and the known class probability matrix O″ to obtain the query set Each query sample The predicted class label of (o j ″ ,1 ,o j ″ ,2 ,…,o j ″ ,C ) represents the probability vector of the j-th row of the matrix O″, Representation sample The predicted category of is unknown, argmax(·) represents the operation of taking the independent variable that maximizes the function value, and the independent variable in the above formula is category c; (4-6) Through each query sample The true category label The unknown class binary probability matrix O′ and the known class probability matrix O″ are used to calculate the query set The classification loss loss3, Among them, a single query sample The classification loss express Belongs to the unknown category.

3. The small sample open set image recognition method based on low-dimensional contrast adaptation as claimed in claim 2, characterized in that: Step (5) is specifically: (5-1) According to the low-dimensional dense feature reconstruction auxiliary loss loss1, the supervised contrast auxiliary loss loss2 and the classification loss loss3, the total model loss loss = loss1 + loss2 + loss3 is obtained, and the small sample open set image recognition model composed of the low-dimensional contrast embedding module, the task adaptive correction module and the metric classification module is optimized by the stochastic gradient descent algorithm. The model is iteratively trained until convergence to obtain the optimized small sample open set recognition model; (5-2) Sampling the new open image set to obtain the support set and an unlabeled query set with unknown categories Input the optimized small sample open set model and execute steps (1) to (4) in sequence to output the query set All samples in The predicted class label of in represents the number of image samples in the support set, Indicates support image samples and corresponding category labels, represents a known set of categories, is the total number of known categories, represents the number of image samples in the query set, Indicates query image samples, For query samples Predict class label, if but show is predicted to be of unknown class.

Images