Image classification method based on continuous adaptive knowledge space

Abstract

The invention discloses an image classification method based on a continuous adaptive knowledge space, and belongs to the field of computer vision and image classification, and the method comprises the steps: constructing a knowledge space of a known sample according to the centroid of a known category and the radius of the centroid, and updating the knowledge space of the known sample to an adaptive knowledge space; in the adaptive knowledge space, calculating a second distance from the mass center of each known category in the current image classification task to all hyperspheres in the knowledge space before updating; and according to the second distance, judging whether the mass center of the hyper-sphere falls in the pseudo-label hyper-sphere, and if so, converting the pseudo-label of the hyper-sphere into a real label. The self-adaptive knowledge space can continuously contain category information of all known samples, and the accuracy of the false label is evaluated by calculating the distance between the centroid of the known category and the hypersphere, so that the false label with high confidence is converted into a real label, and the recognition accuracy of an unknown image and the classification accuracy of a known image are greatly improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and image classification, and in particular to an image classification method based on a continuous adaptive knowledge space. Background Art

[0002] In recent years, deep neural networks have made significant progress in static computer vision tasks. With the development of technology and the expansion of application scenarios, more and more dynamic data flow challenges are faced. In these dynamic environments, traditional unknown image detection methods show their limitations. These methods usually stay at the unknown image detection stage, that is, the existing methods only regard unknown images as a single category and do not further process and transform these images. That is, the existing methods only have the ability to cluster unknown images, but cannot further classify unknown images. In the face of dynamic data streams, the limitation of storage space becomes a key issue. It is obviously unrealistic to store unknown image samples for a long time because it consumes a lot of resources. Therefore, a more efficient strategy is needed to process these unknown samples. At the same time, as training samples of unknown categories gradually appear, the model needs to have the ability to transform unknown categories detected in the past into known categories. How to process unknown samples in real time and transform unknown categories into known categories in real time during continuous learning is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to overcome the problems of the prior art and provide an image classification method based on a continuous adaptive knowledge space.

[0004] The object of the present invention is to achieve the following technical solution: an image classification method based on continuous adaptive knowledge space, the method comprising the following steps: Get the embedded representation of the input image for the current image classification task; Get the representation vector of the embedded representation of the input image; The representation vectors are divided into a positive representation vector set and a negative representation vector set according to the true labels corresponding to the representation vectors, and the centroid of the known categories is constructed according to the positive representation vector set; Calculate the first distance from the negative characterization vector to the centroid in the negative characterization vector set, and construct the radius of each centroid; Constructing a knowledge space of known samples according to the centroid and the radius of the centroid, and updating the knowledge space of the known samples to an adaptive knowledge space; the knowledge space includes the centroid and a hypersphere of the known categories, and the hypersphere determines the boundary and range of each category; In the adaptive knowledge space, the second distance from the centroid of each known category in the current image classification task to all hyperspheres in the knowledge space before updating is calculated; based on the second distance, it is determined whether the centroid of the hypersphere falls within the pseudo-label hypersphere. If so, the pseudo-label of the hypersphere is converted into a true label.

[0005] In one example, before obtaining the embedding representation of the input image of the current image classification task, the method further includes: Construct a prompt pool for the current image classification task. The prompts in the prompt pool are learnable image vectors, and each prompt is associated with a learnable key as a value to form a key-value pair.

[0006] In one example, before obtaining the representation vector of the image embedding representation, the embedded representation is enhanced, specifically including: The similarity distance between the embedding representation and the key in the prompt pool is calculated, and multiple similar keys with similarity distances less than a distance threshold are searched from the prompt pool. The corresponding prompt subsets are matched according to the multiple similar keys, and the matched prompt subsets are concatenated with the image embedding representation to obtain an enhanced image embedding representation.

[0007] In one example, after obtaining the representation vector of the image embedding representation, the method further includes: The representation vector is classified based on the image classification model, and the classification loss function is calculated according to the difference between the classification result and the true label value. The image classification model is back-propagated according to the classification loss function to obtain the optimized prompt pool for the current image classification task, and the prompt pool is updated according to the optimized prompt pool.

[0008] In one example, the knowledge space of known samples is constructed by using the optimized centroid and the optimized radius, and obtaining the optimized centroid and the optimized radius includes: The class average margin loss function is calculated according to the representation vector, centroid and radius. The image classification model is optimized according to the class average margin loss function to obtain the optimized prompt pool, optimized centroid and optimized radius of the current image classification task.

[0009] In one example, obtaining the optimized centroid and optimized radius further includes: Calculate the average distance loss between the embedded representation and multiple similar keys, classify the representation vector based on the image classification model, calculate the classification loss based on the difference between the classification result of the representation vector and the true label value, and determine the loss function based on the data augmentation paradigm based on the average distance loss and classification loss; According to the loss function based on the data augmentation paradigm and the class average margin loss function, the final optimization objective function of the current image classification task is obtained. According to the optimization objective function, the image classification model is optimized to obtain the optimized prompt pool, optimized center of mass and optimized radius of the current image classification task.

[0010] It should be further explained that the technical features corresponding to the various examples of the above method can be combined or replaced with each other to form a new technical solution.

[0011] In one example, an image classification method based on a continuous adaptive knowledge space is used to test an image classification method formed by any one of the above examples or a combination of multiple examples, including the following steps: Get the embedded representation of the image to be tested; Performing enhancement processing on the embedded representation to obtain an enhanced image embedded representation; Get the representation vector of the image embedding representation; In the adaptive knowledge space, the third distance between the representation vector and the centroid of all known categories in the knowledge space before updating is calculated to determine the minimum third distance; the most similar category closest to the representation vector is obtained, and the minimum third distance is compared with the radius of the most similar category. If the minimum third distance is less than the radius of the most similar category, the image to be tested corresponding to the representation vector is classified as the most similar category; otherwise, the image to be tested corresponding to the representation vector is identified as an unknown sample; The unknown samples are clustered to obtain the centroid and radius of each cluster, and a pseudo label is assigned to each cluster and updated to the adaptive knowledge space.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. In one example, the adaptive knowledge space is continuously updated through the knowledge space of known samples. The adaptive knowledge space can continuously contain the category information of all known samples. By calculating the distance between the centroid of the known category and the hypersphere in the knowledge space, it is determined whether the centroid of the hypersphere falls in the pseudo-label hypersphere, and then the accuracy of the pseudo-label is evaluated, so that the high-confidence pseudo-label is converted into a true label. That is, the adaptive knowledge space is used to gradually convert unknown categories into known categories in the process of adapting to the new data distribution, which greatly improves the recognition accuracy of unknown images and the classification accuracy of known images. At the same time, the classification method does not need to rely on a large amount of labeled data and can adapt to application scenarios where samples are scarce.

[0013] 2. In one example, the image classification model is optimized through the class average margin loss function, which can increase the distance between classes and reduce the distance within classes. Continuous unknown image detection based on margins can more effectively reduce the risk of opening and improve the model's ability to recognize and process unknown images.

[0014] 3. In one example, model optimization is performed based on the loss function of the data augmentation paradigm. Based on the suggested data augmentation paradigm, it can overcome the memory overhead and the non-reusability of enhanced samples caused by traditional sample augmentation, effectively solve the overfitting problem caused by small samples, and improve the classification accuracy of known images. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings. The accompanying drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0016] Figure 1 A flow chart of a method provided for an example of the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In the description of the present invention, ordinal numbers (eg, “first and second,” “first to fourth,” etc.) are used to distinguish objects and are not limited to the order and should not be understood as indicating or implying relative importance.

[0019] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, "connected" and "connection" should be understood in a broad sense, for example, it can also be an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] In one example, if Figure 1 As shown, an image classification method based on continuous adaptive knowledge space, which can be used in the training phase of the model, can also be used in the detection and testing phase, and includes the following steps:

[0022] S1: Get the embedded representation of the input image for the current image classification task.

[0023] In this step, the embedding representation of the input image of the current image classification task is obtained through the feature extraction model, such as the Transformer-based image processing model. This example uses the Vision Transformer (ViT) pre-trained based on the ImageNet-1k dataset (a dataset containing 1,000 categories of images), freezes the ViT parameters, and uses it for feature extraction. Specifically, ViT first splits the image into a series of image patches, flattens them, and then inputs them into the Transformer model as a sequence. With the help of the self-attention mechanism and the multi-layer Transformer encoder, ViT can effectively extract image features. ViT includes an image patch embedding module, a position encoding module, a Transformer encoder, a global mean pooling layer, a fully connected layer, and an activation function layer. Among them, the image patch embedding module divides the input image into fixed-size image patches and flattens these patches into vector form; the position encoding module is responsible for encoding the information of each position in the sequence; the Transformer encoder consists of multiple encoding layers, in each encoding layer, the input sequence will first pass through the self-attention layer for feature extraction, and then pass through the fully connected feedforward neural network for feature mapping; the global mean pooling layer obtains the overall feature representation of the image based on the global mean pooling process; the fully connected layer maps the pooled feature vector to the number of category labels, and then outputs the final classification probability through the Softmax activation function layer. This example can obtain the embedded representation of the input image of the current image classification task through the image patch embedding module of ViT.

[0024] S2: Get the representation vector of the embedded representation of the input image.

[0025] Specifically, the representation vector of the embedded representation of the input image is obtained through the feature extraction model, such as the Transformer-based image processing model. This example uses the Vision Transformer (ViT) pre-trained based on the ImageNet-1k dataset, freezes the ViT parameters, and uses it for feature extraction. This example uses the image patch embedding module of ViT to obtain the representation vector of the embedded representation of the input image, that is, the image patch embedding module flattens the image patch into a one-dimensional vector, and converts the one-dimensional vector into an embedding vector of fixed dimension to obtain the representation vector of the embedded representation of the input image.

[0026] S3: The representation vectors are divided into a positive representation vector set and a negative representation vector set according to the true labels corresponding to the representation vectors, and the centroid of the known categories is constructed according to the positive representation vector set.

[0027] Among them, the positive representation vector in the positive representation vector set is the sample vector whose training label matches the current category, and the negative representation vector in the negative representation vector set is the sample vector whose training label does not match the current category; the centroid of the category represents the central feature vector of a category.

[0028] S4: Calculate the first distance from the negative characterization vector to the centroid in the negative characterization vector set, and construct the radius of each centroid;

[0029] In the image classification process, the category boundary can be defined by negative samples (samples corresponding to negative characterization vectors). By calculating the distance from the negative characterization vector to the centroid, the category boundary can be better determined, and then the radius of each centroid can be constructed.

[0030] S5: Construct a knowledge space of known samples according to the centroid and radius, and update the knowledge space of known samples to the adaptive knowledge space.

[0031] The knowledge space includes the centroid and hypersphere of known categories. The hypersphere determines the boundary and range of each category, that is, all samples of a category are located inside the corresponding hypersphere. The adaptive knowledge space also includes the centroid and corresponding hypersphere of known categories, and is continuously updated through the knowledge space of known samples. It can continuously contain the category information of all known samples and thus continuously adapt to new data distribution.

[0032] S6: In the adaptive knowledge space, calculate the second distance from the centroid of each known category in the current image classification task to all hyperspheres in the knowledge space before updating; determine whether the centroid of the hypersphere falls within the pseudo-label hypersphere based on the second distance, and if so, convert the pseudo-label of the hypersphere into a true label.

[0033] When the next new training task arrives, repeat the above steps S1-S6.

[0034] The real label is used to indicate the real category to which the hyperball belongs, and the pseudo label is used to indicate the category to which the hyperball may belong. In this step, by comparing the second distance with the boundary of the pseudo label hyperball, it is possible to determine whether the centroid of the hyperball is located in the pseudo label hyperball. If the centroid of the hyperball falls in the pseudo label hyperball, it means that the pseudo label of the pseudo label hyperball has enough confidence to be considered as the real label, thereby converting the unknown category into a known category, which greatly improves the recognition accuracy of unknown images and the classification accuracy of known images.

[0035] In one example, before obtaining the embedding representation of the input image of the current image classification task, the method further includes: Construct a prompt pool for the current image classification task. The prompts in the prompt pool are learnable image vectors, and each prompt is associated with a learnable key as a value to form a key-value pair.

[0036] Specifically, in the current image classification task During the training process, a task-specific Tips for And perform the initialization operation with uniform distribution: ; ; ; Among them, the prompt pool Contains L key-value pairs, Indicates the key, Represents a value; Indicates the task sequence number; ~ indicates "obeying" a certain data distribution; It represents a uniform distribution from -1 to 1; , All follow a uniform distribution from -1 to 1. Uniform distribution is a commonly used probability distribution. Its key feature is that the probability of each value in the interval appearing is equal, that is, all possible values ​​have the same probability. In addition, other common parameter initialization methods such as zero initialization, normal distribution random initialization, Grote initialization, and He initialization can also be selected. Optionally, for the task , the classification head of ViT is initialized with uniform distribution: ; Among them, the classification head of ViT includes a fully connected layer and a Softmax activation function layer. In the above formula, is the weight matrix to be initialized; represents uniform distribution; Represents the input dimension of the current layer, which is the input dimension of the fully connected layer in this invention.

[0037] In one example, obtaining an embedded representation of an input image of a current image classification task specifically includes: For the current task , training sample set ,in Represents a specific input training sample (input image); Indicates category; For Category The sample set is = ,in, For the current task The number of training samples in this invention Set to 5, the size of a convolution kernel in the pre-trained model (ViT in this example) is The convolutional layer and the size are The flattening layer is used as a query function to encode the input sample set Projected to dimension The vector space , and then concatenate an additional size of of Code (classification mark) and add Positional encoding , after the random dropout layer, we get the input sample set The final embedding representation ,Right now : ; in, Indicates splicing of the same dimension; are the embedding network parameters of the image patch embedding module in ViT; and is the frozen parameter of the ViT model, Represents the category code, Indicates positional encoding; It is a random deactivation layer. It should be noted that the deactivation rate used by the random deactivation layer of the present invention is 0, that is, in the network of the present invention, there is actually no embedding representation deactivation at any position. In actual situations, the deactivation rate can be adjusted according to the environment.

[0038] In one example, before obtaining the representation vector of the image embedding representation, the embedded representation is enhanced, specifically including: Calculate the similarity distance between the embedding representation and the key in the prompt pool, search for multiple similar keys with a similarity distance less than the distance threshold from the prompt pool, match the corresponding prompt subsets according to the multiple similar keys, and concatenate the matched prompt subsets with the image embedding representation to obtain an enhanced image embedding representation. By setting different distance thresholds, different numbers of similar keys can be screened, and the distance threshold can be set based on historical experience or actual usage requirements.

[0039] Specifically, according to the embedding representation , calculated using the cosine similarity distance metric Embedding and Tip Pool The distance between the middle key and the task; based on the similarity distance, the Top-K algorithm (the top K most significant algorithm) is used to select the prompt pool of the current task Search for the first K keys with the smallest distance and match the corresponding hints ; Match the prompts Embedded representation with images Perform same-dimensional splicing to obtain the image embedding representation enhanced by hints ,Right now , the whole calculation specifically includes:

[0040] a. Based on the embedding representation , calculated using the cosine similarity distance metric and Tip Pool The distance between the keys stored in the image embedding representation in the present invention and The cosine similarity distance can be calculated directly: ; in, Indicates cosine similarity calculation; represents the vector modulo operator, For the Embedding Model; Key Of course, other distance measurement methods, such as Euclidean distance, Manhattan distance, etc., can also be used to measure similarity.

[0041] b. Based on the similarity distance obtained in step a, calculate the frequency of selecting each prompt in the prompt pool to obtain a prompt frequency list , For the frequency list quantity label, select the prompt frequency list using the Top-K algorithm from the prompt pool of the current task Search for the first K similar keys with the smallest distance , the Top-K algorithm is as follows: ; in, Indicates the value of the independent variable parameter when the function reaches the minimum value; , M are the quantity labels of the keys, and are the upper and lower limits of the summation function respectively.

[0042] c. Based on the searched top K similar keys Match the corresponding prompt , match the resulting prompts With embedding representation Perform same-dimensional splicing to obtain the image embedding representation enhanced by hints : ; ; in, is a set of serial numbers of K keys.

[0043] In this example, the embedded representation of the image is enhanced to obtain richer image feature information, thereby improving the accuracy of image classification.

[0044] In one example, an enhanced image embedding representation is obtained through the visual self-attention mechanism ViT. The representation vector : , in, ; It is the encoder (parameter freezing) in the ViT model, including the position encoding module, Transformer encoder, etc.

[0045] In one example, constructing the centroid of a known category specifically includes:

[0046] The representation vector enhanced by the acquired hint and the corresponding true label Based on this, we can construct a set of known categories The centroid , is the number of categories. For each category , input training sample set Divided into positive sample sets and negative sample set ,Right now: ; ; in, represents a specific input training sample, Represents the true label (true category label) corresponding to the input training sample; Represents the input training sample set; Indicates known categories;

[0047] Furthermore, the representation vector enhanced by the hint Divided into positive characterization vector sets and negative representation vector set , the formula for constructing the centroid is as follows: ; In the above formula, For Category The centroid of is the number of positive characterization vectors; Represents the positive characterization vector set A representation vector in .

[0048] In one example, the radius of each centroid is constructed, including: In the calculation of the centroid set Based on this, the negative characterization vector is calculated To the centroid The Euclidean distance of , the number of negative characterization vectors is much greater than the positive characterization vectors, and the calculation expression for constructing the radius of each centroid is as follows: ; In the above formula, represents the radius of the center of mass; is the quantile function; is the Euclidean distance; is a hyperparameter that controls the robustness of the radius, is a hyperparameter that controls the margin between classes.

[0049] In one example, after obtaining the representation vector of the image embedding representation or constructing the radius of each centroid, the method further includes:

[0050] Based on the image classification model (ViT), the representation vector is classified and processed, and the classification loss function is calculated according to the difference between the classification result and the true label value. The image classification model is back-propagated and optimized according to the classification loss function to obtain the optimized prompt pool of the current image classification task, and the prompt pool is updated according to the optimized prompt pool.

[0051] In this example, a prompt pool including the learnable parameters of the model is constructed, and the model is optimized through the classification loss function, and then the model parameters are updated to improve the classification performance of the model.

[0052] In one example, the knowledge space of known samples is constructed by using the optimized centroid and the optimized radius, and obtaining the optimized centroid and the optimized radius includes:

[0053] The class average margin loss function is calculated according to the representation vector, centroid and radius. The image classification model is optimized according to the class average margin loss function to obtain the optimized prompt pool, optimized centroid and optimized radius of the current image classification task.

[0054] Preferably, obtaining the optimized centroid and optimized radius further includes:

[0055] Calculate the average distance loss between the embedded representation and multiple similar keys, classify the representation vector based on the image classification model, calculate the classification loss based on the difference between the classification result of the representation vector and the true label value, and determine the loss function based on the data augmentation paradigm based on the average distance loss and classification loss;

[0056] The class average margin loss function is calculated according to the representation vector, centroid and radius. The final optimization objective function of the current image classification task is obtained according to the loss function based on the data augmentation paradigm and the class average margin loss function. The image classification model is optimized according to the optimization objective function to obtain the optimized prompt pool, optimized centroid and optimized radius of the current image classification task.

[0057] In this example, the image classification model optimization process based on the optimization objective function specifically includes:

[0058] A. Computing Embedded Representations and Tip Pool Similar key The average distance loss , the calculation expression is: ; in, Indicates Similar keys.

[0059] B. Output representation vector through ViT’s fully connected layer The classification results and the true label values ​​are calculated. The difference between , using the cross entropy loss function to calculate the classification loss , the calculation expression is: ; In the above formula, represents the cross entropy loss function; For classification head; , For sample The true probability of belonging to category C is When the category is C, it takes 1, otherwise it takes 0; For sample The predicted probability of belonging to class C.

[0060] C. Based on the average distance loss and classification loss , constructing a loss function for the hint-based data augmentation paradigm : ; in, is the hyperparameter of the average distance loss. Loss function based on the data augmentation paradigm Optimize the image classification model, and the optimization objective function is: ; in, are the learnable parameters of the model.

[0061] D. Based on the characterization vector , Centroid , the radius of the center of mass , computing tasks All categories The class average margin loss function is : ; In the formula, N is the task The number of categories in the training set; For Category The radius of For Category The set of positive characterization vectors of ; For Category The negative representation vector set of ; For Category The centroid of is the margin between classes; is the constraint radius size hyperparameter, , is the balance scale factor between the positive and negative characterization vector sets.

[0062] E. Loss Function for Data Augmentation Paradigm And the class average margin loss function Perform weighted processing to obtain the task The final optimization objective function is: , in, for and According to the optimization objective function, the present invention uses the Adam optimizer to perform back propagation update. Of course, other optimization methods, such as stochastic gradient descent, etc., can also be used.

[0063] F. According to the final optimization objective function, the optimization model obtains the task Optimized centroids and optimized radii of the final hint pool and known samples.

[0064] Optionally, based on the optimized task Tips pool, update , through continuous accumulation, a constantly updated prompt pool can be obtained, and the update calculation formula is as follows:

[0065] When t=1, then Initially empty.

[0066] In one example, the knowledge space of known samples is constructed based on the centroid and the radius of the centroid. , updating the knowledge space of known samples to the adaptive knowledge space, including:

[0067] First, construct the knowledge space of each category based on the optimized centroid and radius, that is, the knowledge space of known samples ;

[0068] Then update the knowledge space of known samples to the adaptive knowledge space, that is:

[0069] The left side of the equation is the updated adaptive knowledge space, and the right side of the equation It is the knowledge space before the update; Indicates known categories; Represents the known categories of the current image classification task; is the set of centroids in the knowledge space (before updating); Represents the set of known category centroids in the current image classification task; The set of radii representing the center of mass; Represents the radius set of the centroid in the current image classification task.

[0070] In one example, according to the updated adaptive knowledge space, based on the assumption, there is a pseudo-label in the space Super Ball , if a hypersphere The centroid Falling on the pseudo label Super Ball , then the pseudo label Convert to true label , specifically including:

[0071] First, according to the adaptive knowledge space , calculate the current task The centroid of each known sample in The distance to all hyperspheres in the knowledge space (before updating) (second distance), expressed as hypersphere For example ( ),Right now: ; in, , for the super sphere Center of mass arrive The set of distances to all centroids in .

[0072] Then according to the calculated distance , assuming that there is a pseudo label in the space Super Ball , if the super ball The center of mass Falling on the super ball Middle, that is, distance Smaller than radius , then the pseudo label Convert to true label ,Right now: .

[0073] The above examples are combined to obtain a preferred example of the present invention, in which the method comprises the following steps:

[0074] S10: construct a prompt pool for the current image classification task. The prompt pool includes multiple key-value pairs. The key-value pairs are learnable parameters, including weights and biases of the image classification model ViT.

[0075] S20: Obtain the embedded representation of the input image of the current image classification task through ViT;

[0076] S30: calculating the similarity distance between the embedding representation and the key in the prompt pool, searching for multiple similar keys whose similarity distance is less than a distance threshold from the prompt pool, matching corresponding prompt subsets according to the multiple similar keys, and concatenating the matched prompt subsets with the image embedding representation to obtain an enhanced image embedding representation;

[0077] S40: Obtain the representation vector of the image embedding representation through ViT;

[0078] S50: dividing the representation vector into a positive representation vector set and a negative representation vector set according to the true label corresponding to the representation vector, and constructing the centroid of the known category according to the positive representation vector set;

[0079] S60: Calculate the first distance from the negative characterization vector in the negative characterization vector set to the centroid, and construct the radius of each centroid;

[0080] S70: Calculate the average distance loss between the embedded representation and multiple similar keys, where the similar keys are multiple keys whose similarity distance to the embedded representation is less than a distance threshold; classify the representation vector based on the image classification model, calculate the classification loss according to the difference between the classification result of the representation vector and the true label value, and determine the loss function based on the data enhancement paradigm according to the average distance loss and the classification loss; calculate the class average margin loss function according to the representation vector, the centroid and the radius, obtain the final optimization objective function of the current image classification task according to the loss function based on the data enhancement paradigm and the class average margin loss function, optimize the image classification model according to the optimization objective function to obtain the optimized prompt pool, optimized centroid and optimized radius of the current image classification task, and update the prompt pool according to the optimized prompt pool;

[0081] S80: constructing a knowledge space of known samples according to the optimized centroid and the optimized radius, and updating the knowledge space of the known samples to the adaptive knowledge space;

[0082] S90: In the adaptive knowledge space, calculate the second distance from the centroid of each known category in the current image classification task to all hyperspheres in the knowledge space before updating; determine whether the centroid of the hypersphere falls within the pseudo-label hypersphere based on the second distance, and if so, convert the pseudo-label of the hypersphere into a true label.

[0083] The present invention designs a strategy for transforming unknown categories into known categories, that is, classifying unknown samples based on a continuously updated adaptive knowledge space, aiming to process unknown samples in real time, transform unknown categories into known categories, and ensure that the model can adapt to new data distributions in a timely manner during continuous learning, thereby improving the classification performance of the model. At the same time, the present invention performs continuous unknown image detection based on margins, which can more effectively reduce open risks and improve the model's ability to recognize and process unknown images. The present invention also designs a prompt-based data enhancement paradigm, which overcomes the problems of memory overhead and non-reusability of enhanced samples caused by traditional sample enhancement, can effectively solve the overfitting problem caused by small samples, and improve the classification performance of known images.

[0084] After the image classification model is trained according to the above preferred method, the model is used to perform unknown detection and classification. At this time, testing the image classification method includes the following steps: S100: Obtain image data to be tested through trained ViT = Embedding Representation , The number of image data for testing; S200: Embedding Representation Perform enhancement processing to obtain enhanced image embedding representation ; S300: Obtaining image embedding representation through ViT The representation vector ; S400: Calculate the representation vector in the adaptive knowledge space The distance to the centroid of all known categories in the knowledge space , according to the calculated minimum distance , get the distance The smallest class and its radius There are two cases for the comparison results: (1) Less than , then the sample Classified as ; (2) If Greater than , it is identified as an unknown sample.

[0085] S500: Cluster the unknown samples to obtain the centroid and radius of each cluster, assign a pseudo label to each cluster and update it to the adaptive knowledge space.

[0086] Furthermore, the specific implementation of each sub-step of step S400 is as follows:

[0087] 1) In the adaptive knowledge space, calculate the third distance between the representation vector and the centroid of all known categories in the knowledge space before updating, and determine the minimum third distance (minimum distance );

[0088] Specifically, the image representation vector enhanced by the hint ,calculate The distance to the centroid of all known categories in the knowledge space (Third distance): , in, is the set of centroids of known categories in the knowledge space, is the centroid arrive The set of distances to all centroids in .

[0089] 2) Get the most similar category closest to the distance representation vector, and the minimum distance Compare the size with the radius of the most similar category. If the minimum distance If the radius is smaller than the most similar category, the sample corresponding to the representation vector is classified as the most similar category; otherwise, the sample corresponding to the representation vector is identified as an unknown sample;

[0090] Specifically, get the distance centroid The smallest class (most similar category), and for the minimum distance With class Radius To compare the size: like Less than , then the sample Classified as : ; like Greater than , then the sample Identified as unknown: .

[0091] 3) Cluster the unknown samples, obtain the centroid and radius of each cluster, and assign a pseudo label to each cluster , and update it to the adaptive knowledge space:

[0092] in, Represents the category defined by the pseudo label; represents the set of centroids defined by pseudo labels; Represents a collection of radii defined by pseudo labels.

[0093] The above specific implementation methods are detailed descriptions of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. An image classification method based on a continuous adaptive knowledge space, characterized in that: The following steps are involved: Get the embedded representation of the input image for the current image classification task; Get the representation vector of the embedded representation of the input image; The representation vectors are divided into a positive representation vector set and a negative representation vector set according to the true labels corresponding to the representation vectors, and the centroid of the known categories is constructed according to the positive representation vector set; Calculate the first distance from the negative characterization vector to the centroid in the negative characterization vector set, and construct the radius of each centroid; Construct a knowledge space of known samples according to the centroid and the radius of the centroid, and update the knowledge space of known samples to the adaptive knowledge space; The knowledge space includes centroids of known categories and hyperspheres, the hyperspheres determining the boundaries and range of each category; In the adaptive knowledge space, the second distance from the centroid of each known category in the current image classification task to all hyperspheres in the knowledge space before updating is calculated; based on the second distance, it is determined whether the centroid of the hypersphere falls within the pseudo-label hypersphere. If so, the pseudo-label of the hypersphere is converted into a true label.

2. The image classification method based on continuous adaptive knowledge space according to claim 1 is characterized in that: Before obtaining the embedded representation of the input image of the current image classification task, it also includes: Construct a prompt pool for the current image classification task. The prompts in the prompt pool are learnable image vectors, and each prompt is associated with a learnable key as a value to form a key-value pair.

3. The image classification method based on continuous adaptive knowledge space according to claim 2 is characterized in that: Before obtaining the representation vector of the image embedding representation, the embedding representation is also enhanced, including: The similarity distance between the embedding representation and the key in the prompt pool is calculated, and multiple similar keys with similarity distances less than a distance threshold are searched from the prompt pool. The corresponding prompt subsets are matched according to the multiple similar keys, and the matched prompt subsets are concatenated with the image embedding representation to obtain an enhanced image embedding representation.

4. The image classification method based on continuous adaptive knowledge space according to claim 2, characterized in that: After obtaining the representation vector of the image embedding representation, it also includes: The representation vector is classified based on the image classification model, and the classification loss function is calculated according to the difference between the classification result and the true label value. The image classification model is back-propagated according to the classification loss function to obtain the optimized prompt pool for the current image classification task, and the prompt pool is updated according to the optimized prompt pool.

5. The image classification method based on continuous adaptive knowledge space according to claim 2 is characterized in that: The knowledge space of known samples is constructed by using the optimized centroid and optimized radius. The optimized centroid and optimized radius are obtained by: The class average margin loss function is calculated according to the representation vector, centroid and radius. The image classification model is optimized according to the class average margin loss function to obtain the optimized prompt pool, optimized centroid and optimized radius of the current image classification task.

6. The image classification method based on continuous adaptive knowledge space according to claim 5, characterized in that: Obtaining the optimized centroid and radius also includes: Calculate the average distance loss between the embedded representation and multiple similar keys, classify the representation vector based on the image classification model, calculate the classification loss based on the difference between the classification result of the representation vector and the true label value, and determine the loss function based on the data augmentation paradigm based on the average distance loss and classification loss; According to the loss function based on the data augmentation paradigm and the class average margin loss function, the final optimization objective function of the current image classification task is obtained. According to the optimization objective function, the image classification model is optimized to obtain the optimized prompt pool, optimized center of mass and optimized radius of the current image classification task.

7. An image classification method based on a continuous adaptive knowledge space, characterized in that: Used to test the method according to any one of claims 1 to 6, comprising the following steps: Get the embedded representation of the image to be tested; Performing enhancement processing on the embedded representation to obtain an enhanced image embedded representation; Get the representation vector of the image embedding representation; In the adaptive knowledge space, the third distance between the representation vector and the centroid of all known categories in the knowledge space before updating is calculated to determine the minimum third distance; the most similar category closest to the representation vector is obtained, and the minimum third distance is compared with the radius of the most similar category. If the minimum third distance is less than the radius of the most similar category, the image to be tested corresponding to the representation vector is classified as the most similar category; otherwise, the image to be tested corresponding to the representation vector is identified as an unknown sample; The unknown samples are clustered to obtain the centroid and radius of each cluster, and a pseudo label is assigned to each cluster and updated to the adaptive knowledge space.

Images