Image classification and model training method, device and equipment, and storage medium

Abstract

Embodiments of the present application disclose an image classification and model training method, device and equipment, and a storage medium, wherein the image classification method comprises: obtaining a first image to be classified; performing feature extraction on a set support image set and the first image respectively to obtain a support feature set and a first image feature; wherein the support image set comprises at least one support image corresponding to each classification category, and the support feature set comprises support features corresponding to each classification category; updating each support feature and the first image feature based on the similarity between each support feature and the first image feature to obtain updated each support feature and updated first image feature; and classifying the first image based on the similarity between updated each support feature and updated first image feature to obtain the classification category of the first image.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of deep learning technology in computers, and in particular to an image classification and model training method, apparatus and device, and storage medium. Background Technology

[0002] Few-shot learning aims to develop the learning ability of deep learning models in scenarios with limited samples. Few-shot image classification is a fundamental task in few-shot learning, with the goal of determining the category of an image based on a small number of labeled images. However, the classification accuracy of few-shot image classification in related technologies is not high. Summary of the Invention

[0003] In view of this, embodiments of this application provide at least one image classification method and model training method, apparatus and device, storage medium and program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide an image classification method, the method comprising:

[0006] Obtain the first image to be classified;

[0007] Feature extraction is performed on the set of support images and the first image to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories;

[0008] Based on the similarity between each of the supporting features and the first image feature, each of the supporting features and the first image feature is updated to obtain each of the supporting features and the first image feature after the update.

[0009] Based on the similarity between each of the updated supporting features and the updated first image features, the first image is classified to obtain the classification category of the first image.

[0010] On the other hand, embodiments of this application provide a model training method, the method comprising:

[0011] Obtain a second image with category labels;

[0012] Using the first feature extraction network, features are extracted from the set of support images to obtain a support feature set; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories;

[0013] The second feature extraction network is used to extract features from the second image to obtain the second image features corresponding to the second image.

[0014] Based on the similarity between each of the supporting features and the second image features, each of the supporting features and the second image features is updated to obtain updated supporting features and updated second image features;

[0015] A loss value is determined based on each of the updated supporting features, the updated second image features, and the category label;

[0016] Based on the loss value, the model is updated at least once to obtain the trained model.

[0017] Furthermore, embodiments of this application provide an image classification apparatus, the apparatus comprising:

[0018] The acquisition module is used to acquire the first image to be classified.

[0019] The first extraction module is used to extract features from the set of support images and the first image respectively to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories;

[0020] The first update module is used to update each of the supporting features and the first image features based on the similarity between each of the supporting features and the first image features, so as to obtain each of the supporting features and the first image features after the update.

[0021] The first classification module is used to classify the first image based on the similarity between each of the updated supporting features and the updated first image features, so as to obtain the classification category of the first image.

[0022] In another aspect, embodiments of this application provide a model training apparatus, the apparatus comprising:

[0023] The second acquisition module is used to acquire a second image with category labels;

[0024] The second extraction module is used to extract features from a set of support images using a first feature extraction network to obtain a support feature set; wherein the support image set includes at least one support image corresponding to each of multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories.

[0025] The third extraction module is used to extract features from the second image using the second feature extraction network to obtain the second image features corresponding to the second image.

[0026] The second update module is used to update each of the supporting features and the second image features based on the similarity between each of the supporting features and the second image features, so as to obtain each of the supporting features and the second image features after the update.

[0027] The determination module is used to determine a loss value based on each of the updated supporting features, the updated second image features, and the category label;

[0028] The third update module is used to update the model at least once based on the loss value to obtain the trained model.

[0029] In another aspect, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0030] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0031] In another aspect, embodiments of this application provide a computer program including computer-readable code, wherein when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps for implementing the above-described method.

[0032] In another aspect, embodiments of this application provide a computer program product, the computer program product including a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is read and executed by a computer, it implements some or all of the steps in the above method.

[0033] In this embodiment, a first image to be classified is acquired; features are extracted from a set of supporting images and the first image to obtain a supporting feature set and features of the first image; based on the similarity between each supporting feature in the supporting feature set and the features of the first image, each supporting feature and the features of the first image are updated to obtain updated supporting features and updated features of the first image; based on the similarity between the updated supporting features and the updated features of the first image, the first image is classified to obtain the classification category of the first image. Thus, by updating each supporting feature and the features of the first image based on the similarity between each supporting feature and the features of the first image, the correlation between each supporting feature and the features of the first image can be strengthened, deepening the understanding of the supporting images and the first image. Therefore, classifying the first image based on the similarity between the updated supporting features and the updated features of the first image can improve the classification accuracy of the first image.

[0034] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0036] Figure 1 A schematic diagram illustrating the implementation process of an image classification method provided in this application embodiment;

[0037] Figure 2 A schematic diagram illustrating the implementation process of an image classification method provided in this application embodiment;

[0038] Figure 3 A schematic diagram illustrating the implementation process of a model training method provided in this application embodiment;

[0039] Figure 4A A schematic diagram illustrating the implementation process of an image classification method provided in this application embodiment;

[0040] Figure 4B This is a schematic diagram of the composition structure of an image classification system provided in an embodiment of this application;

[0041] Figure 5A This is a schematic diagram of the composition structure of an image classification device provided in an embodiment of this application;

[0042] Figure 5B This is a schematic diagram of the composition structure of a model training device provided in an embodiment of this application;

[0043] Figure 6This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0047] In related technologies, the goal of few-shot image classification is to obtain the classification category of an input image based on a small number of labeled images. However, these technologies do not adequately explore the correlation between few-shot data, resulting in a lack of in-depth understanding of the few-shot data during the few-shot learning process. Consequently, the performance of few-shot image classification is unsatisfactory.

[0048] This application provides an image classification method that can be executed by a processor of a computer device. The computer device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device). Figure 1 This is a schematic diagram illustrating the implementation process of an image classification method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps S101 to S104:

[0049] Step S101: Obtain the first image to be classified.

[0050] Here, the first image to be classified can be any suitable image to be classified, including but not limited to at least one of images of people, animals, vehicles, or specific objects.

[0051] In some implementations, those skilled in the art can obtain a suitable first image based on the actual application scenario, and the embodiments of this application are not limited thereto. For example, the first image can be obtained from at least one of the following: images captured by a camera, video or image frames in an image frame sequence, medical images, images generated using computer technology, etc.

[0052] Step S102: Extract features from the set support image set and the first image respectively to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories.

[0053] Here, the set of supporting images can correspond to multiple category labels, and each category can correspond to at least one supporting image belonging to that category. For example, the multiple category labels corresponding to the supporting image set include Golden Retriever, Samoyed, and Miniature Pinscher, and the supporting image set can include at least one supporting image for the Golden Retriever category, at least one supporting image for the Samoyed category, and at least one supporting image for the Miniature Pinscher category. Here, the supporting images corresponding to different categories in the supporting image set can serve as classification reference standards to guide the classification of the first image.

[0054] In some implementations, deep neural networks or other machine learning methods can be used to extract features from the set of supporting images and the first image respectively, to obtain a supporting feature set and features of the first image. For example, convolutional neural networks (CNNs) can be used to extract features from the set of supporting images and the first image respectively, to obtain a supporting feature set and features of the first image.

[0055] In some implementations, the supporting features and the first image features have multiple representation forms. Those skilled in the art can use any suitable feature representation method to represent the supporting features and the first image features, and the embodiments of this application are not limited in this regard. For example, the supporting features can be represented by support vectors, and the first image features can be represented by query vectors.

[0056] Step S103: Based on the similarity between each of the supporting features and the first image feature, update each of the supporting features and the first image feature to obtain updated supporting features and updated first image features.

[0057] Here, similarity is a measure of the degree of similarity between the supporting image and the first image. The more similar the supporting image and the first image are, the greater the similarity between the supporting image features and the first image features; conversely, the less similar the supporting image and the first image are, the smaller the similarity between the supporting image features and the first image features. For example, if the first image is a golden retriever image, then supporting images in the supporting image set that classify golden retrievers are more similar to the first image, and the similarity between the supporting image features classifying golden retrievers and the first image features classifying golden retrievers is also greater. In some implementations, the similarity calculation method between each supporting feature and the first image feature may include, but is not limited to, at least one of cosine similarity, Euclidean distance, Pearson correlation coefficient, attention-based scoring, K-nearest neighbor algorithm, etc.

[0058] Step S104: Based on the similarity between each of the updated supporting features and the updated first image features, classify the first image to obtain the classification category of the first image.

[0059] In practice, those skilled in the art can classify the first image based on the similarity between each updated supporting feature and the updated first image features, according to the actual situation, and there are no limitations here.

[0060] In some implementations, each updated support feature can be sorted according to its similarity to determine the second target support feature with the highest similarity to the updated first image feature; the classification category corresponding to the second target support feature is then determined as the classification category of the first image.

[0061] In some implementations, a similarity threshold can be set, and a third target support feature can be selected from at least one support feature whose similarity to the updated first image feature is greater than the similarity threshold. The classification category corresponding to the third target support feature is then determined as the classification category of the first image.

[0062] In this embodiment, the supporting features and the first image features are updated based on the similarity between the supporting features and the first image features. Then, the first image is classified based on the similarity between the updated supporting features and the updated first image features. This strengthens the correlation between each supporting feature and the first image features, deepens the understanding of the supporting image and the first image, and thus improves the classification accuracy of the first image.

[0063] In some embodiments, the feature extraction of the set support image set and the first image in step S102 above to obtain the support feature set and the first image features may include steps S111 to S112:

[0064] Step S111: For each of the classification categories, the first feature extraction network is used to extract features from at least one support image in the support image set corresponding to the classification category, so as to obtain the support features corresponding to the classification category.

[0065] Here, the first feature extraction network is a pre-trained feature extraction network used to extract features from the image. The feature extraction network is the basic feature extractor for object classification tasks; its main task is to take the image as input and output a feature map of the corresponding input image.

[0066] In some implementations, when a classification category corresponds to a support image, the image features extracted from that support image can be used as the support features corresponding to that classification category.

[0067] In some implementations, when a certain category corresponds to multiple support images, the multiple support images can be fused first and then feature extraction can be performed to obtain the support features corresponding to the category. In some implementations, when a certain category corresponds to multiple support images, the features of each support image can be extracted separately, and then the features can be fused to obtain the support features corresponding to the category.

[0068] Step S112: Use the second feature extraction network to extract features from the first image to obtain the features of the first image.

[0069] In some implementations, the second feature extraction network is the same as the first feature extraction network, and the two share weights.

[0070] During implementation, after the support image set and the first image are processed by their respective feature extraction networks, they will each obtain the support feature set and the first image features.

[0071] In this embodiment, for each classification category, a first feature extraction network is used to extract features from at least one support image in the support image set corresponding to the classification category, obtaining the support features corresponding to the classification category; a second feature extraction network is used to extract features from the first image, obtaining the first image features. Thus, using two feature extraction networks to obtain the support features and the first image features respectively facilitates the separate learning of the support features and the first image features, thereby deepening the understanding of the support images and the first image during the classification process, and ultimately improving the accuracy of image classification.

[0072] In some embodiments, the classification of the first image based on the similarity between each updated supporting feature and the updated first image feature, as described in step S104 above, to obtain the classification category of the first image, may include steps S121 to S122:

[0073] Step S121: From each of the updated support features, determine the second target support feature that has the highest similarity to the updated first image feature.

[0074] Step S122: Determine the classification category corresponding to the second target supporting feature as the classification category of the first image.

[0075] Here, the support features are associated with predefined category labels, which are used to determine the classification category of the support features. In some implementations, the classification category of the first image can be obtained based on the updated category label of the support feature that has the highest similarity to the first image features.

[0076] In the above embodiments, by selecting the classification category of the updated supporting feature that has the highest similarity to the features of the first image, the most likely classification category of the first image can be obtained, thereby improving the accuracy of image classification.

[0077] This application provides an image classification method, which can be executed by a computer device's processor. For example... Figure 2 As shown, the method includes the following steps S201 to S205:

[0078] Step S201: Obtain the first image to be classified.

[0079] Step S202: Extract features from the set support image set and the first image respectively to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories.

[0080] Here, steps S201 and S202 can be implemented with reference to the specific implementation methods of steps S101 and S102 described above.

[0081] Step S203: Based on the similarity between at least one of the supporting features and the first image feature, update the first image feature to obtain the updated first image feature.

[0082] Step S204: For each of the supporting features, the supporting features are updated based on the similarity between the supporting features and the first image features before the update, to obtain the updated supporting features.

[0083] Step S205: Based on the similarity between each of the updated supporting features and the updated first image features, classify the first image to obtain the classification category of the first image.

[0084] Here, step S205 can be implemented with reference to the specific implementation of step S105 described above.

[0085] In this embodiment, the first image feature is updated based on the similarity between at least one supporting feature and the first image feature to obtain the updated first image feature; for each supporting feature, the supporting feature is updated based on the similarity between the supporting feature and the first image feature before the update to obtain the updated supporting feature. This strengthens the correlation between each supporting feature and the first image feature, deepens the understanding of the supporting image and the first image, and thus improves the accuracy of image classification.

[0086] In some embodiments, updating the first image feature based on the similarity between at least one of the supporting features and the first image feature to obtain the updated first image feature in step S203 above may include steps S211 to S213:

[0087] Step S211: Based on the similarity between each of the supporting features and the first image feature, determine at least one first target supporting feature from the set of supporting features.

[0088] In some implementations, those skilled in the art can determine at least one first target support feature based on similarity according to the actual application scenario, and the embodiments of this application are not limited thereto. For example, at least one first target support feature with the highest similarity to the first image feature can be selected, or at least one first target support feature with a similarity greater than a preset threshold can be selected.

[0089] Step S212: Based on the similarity between each first target support feature and the first image feature, perform a weighted summation on each first target support feature to obtain the first updated feature.

[0090] Here, for each first target support feature, the similarity between the first target support feature and the first image feature can be used as the weight of the first target support feature. Based on the weight of each first target support feature, a weighted sum is performed on all first target support features to obtain the first updated feature. It can be understood that the first target support feature with a greater similarity to the first image feature has a greater weight in the calculation of the first updated feature, and the first target support feature with a smaller similarity to the first image feature has a smaller weight in the calculation of the first updated feature.

[0091] In some implementations, the similarity between the first target supporting features and the first image features can be obtained; for the first image features, each first target supporting feature is sorted according to its similarity to the first image features; a threshold M is set, and the top M first target supporting features in the similarity ranking are selected, and the weighted sum of these M first target supporting features is obtained to obtain the first updated feature. Here, M is a positive integer.

[0092] In implementation, the M first target support features L1, L2, ... L1 with the highest similarity to the first image features can be obtained. M The M first target support features L1, L2, ... L M The similarity scores between the features and the first image features are B1, B2, ..., B1, B2, ..., B2 ... M The first updated feature V can be obtained by weighted summation of the M first target support features. n For example, the first updated feature V can be obtained using the method shown in Formula 1 below. n :

[0093] V n =B1*L1+B2*L2+……+B M *L M (1).

[0094] Step S213: Based on the first updated feature, update the first image feature to obtain the updated first image feature.

[0095] In implementation, any suitable method can be used, based on the first update feature V. n For the first image feature V c The updated first image feature V is obtained by performing an update. cnew This application does not limit the scope of the embodiments. For example, the updated first image feature V can be obtained using the method shown in Formula 2 below. cnew :

[0096] V cnew =Vc +V n =V c +B1*L1+B2*L2+……+B M *L M (2).

[0097] In the above embodiments, at least one first target support feature is determined from the support feature set based on the similarity between each support feature and the first image feature; each first target support feature is weighted and summed based on the similarity between each first target support feature and the first image feature to obtain a first updated feature; the first image feature is updated based on the first updated feature to obtain the updated first image feature. Thus, in the process of calculating the first updated feature, the first target support feature with a greater similarity to the first image feature has a larger weight, and the first target support feature with a smaller similarity to the first image feature has a smaller weight. Therefore, in the process of updating the first image feature based on the first updated feature, the first target support feature with a greater similarity to the first image feature has a greater influence, and the first target support feature with a smaller similarity to the first image feature has a smaller influence. In this way, the correlation between the first target support features with high similarity to the first image features before the update and the updated first image features can be enhanced, while the correlation between the first target support features with low similarity to the first image features before the update and the updated first image features can be weakened. In other words, the correlation between each support feature and the first image features can be strengthened, thereby deepening the understanding of the first image during the classification process and improving the accuracy of image classification.

[0098] In some embodiments, the number of the first images is multiple; the step S204 above, which updates the support features based on the similarity between the support features and the features of the first image before the update, to obtain the updated support features, may include steps S221 to S223:

[0099] Step S221: Based on the similarity between the first image feature corresponding to each first image and the supporting feature, at least one target image feature is determined.

[0100] In some implementations, those skilled in the art can determine at least one first target image feature based on similarity according to the actual application scenario, and the embodiments of this application are not limited thereto. For example, at least one first target image feature with the highest similarity to the supporting feature can be selected, or at least one first target image feature with a similarity greater than a preset threshold can be selected.

[0101] Step S222: Based on the similarity between each target image feature and the supporting features, a weighted sum is performed on each target image feature to obtain the second updated feature.

[0102] Here, for each target image feature, the similarity between that target image feature and its supporting features can be used as the weight of that target image feature. Based on these weights, a weighted sum of all target image features is calculated to obtain the second updated feature. It can be understood that target image features with greater similarity to their supporting features have a larger weight in the calculation of the second updated feature, while target image supporting features with smaller similarity to their supporting features have a smaller weight.

[0103] In some implementations, the similarity between each target image feature and supporting features can be obtained; for supporting features, each target image feature is sorted according to its similarity to the supporting features; a threshold K is set, and the top K target image features in the similarity ranking are selected, and the weighted sum of these K target image features is performed to obtain the second updated feature. Here, K is a positive integer.

[0104] During implementation, the K target image features T1, T2, ... T1 with the highest similarity to the supporting features can be obtained. K The K target image features T1, T2, ... T K The similarities between the features and the supporting features are A1, A2, ..., A K The second updated feature Q can be obtained by weighted summation of the K target image features. n For example, the second updated feature Q can be obtained using the method shown in Formula 3 below. n :

[0105] Q n =A1*T1+A2*T2+……+A K *T K (3).

[0106] Step S223: Based on the second updated feature, update the supporting feature to obtain the updated supporting feature.

[0107] In implementation, any suitable method can be used, based on the second update feature Q. n For supporting feature Q c Perform an update to obtain the updated supported feature Q. cnew This application does not limit the scope of the embodiments. For example, the updated support feature Q can be obtained using the method shown in Formula 4 below. cnew :

[0108] Q cnew =Q c +Q n =Q c +A1*T1+A2*T2+……+A K *T K (4).

[0109] In the above embodiments, at least one target image feature is determined based on the similarity between the first image feature corresponding to each first image and the supporting features. Based on the similarity between each target image feature and the supporting features, each target image feature is weighted and summed to obtain a second updated feature. Thus, in the calculation of the second updated feature, the target image feature with a higher similarity to the supporting features has a higher weight, and the target image feature with a lower similarity to the supporting features has a lower weight. Therefore, in the process of updating the supporting features based on the second updated feature, the target image feature with a higher similarity to the supporting features has a greater influence, and the target image feature with a lower similarity to the supporting features has a smaller influence. This strengthens the correlation between target image features with high similarity to the original target features and the updated supporting features, while weakening the correlation between target image features with low similarity to the original supporting features and the updated supporting features. In other words, it strengthens the correlation between each supporting feature and the first image feature, thereby deepening the understanding of the supporting images during the classification process and improving the accuracy of image classification.

[0110] This application provides a model training method, wherein the model includes a first feature extraction network and a second feature extraction network, and the method can be executed by a processor of a computer device. Figure 3 As shown, the method includes the following steps S301 to S306:

[0111] Step S301: Obtain a second image with category labels.

[0112] Here, the second image with category labels can be read from a pre-defined sample image library, downloaded from the internet, or obtained by manually or using computer equipment to label at least one second image category. Category labels include various categories. In implementation, those skilled in the art can use appropriate methods to obtain the sample image set according to the actual situation; this application embodiment is not limited in this regard.

[0113] In some implementations, the category label for each second image can be manually labeled or automatically labeled by a computer. The category label includes a label representing the category of the object. For example, a second image corresponding to a Golden Retriever would have the category label "Golden Retriever."

[0114] Step S302: Using the first feature extraction network, feature extraction is performed on the set of support images to obtain a support feature set; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories.

[0115] In some implementations, when a category corresponds to a single supporting image, the image features extracted from that supporting image can be used as the supporting features corresponding to that category. For example, in a set of supporting images, if there is only one corresponding Golden Retriever image for the Golden Retriever category, the image features extracted from that Golden Retriever image can be used as the supporting features corresponding to the Golden Retriever category.

[0116] In some implementations, when a category corresponds to multiple supporting images, the multiple supporting images can be fused first, and then feature extraction can be performed to obtain the supporting features corresponding to that category. In other implementations, when a category corresponds to multiple supporting images, the features of each supporting image can be extracted separately, and then the features can be fused together to obtain the supporting features corresponding to that category. For example, in a support image set, if there are multiple corresponding Golden Retriever images for the category of Golden Retriever, the multiple Golden Retriever images can be fused into a single Golden Retriever image, and then feature extraction can be performed on this Golden Retriever image to obtain the supporting features corresponding to that category. Alternatively, the image features of multiple Golden Retriever images can be extracted separately, and then the multiple features can be fused together to obtain the supporting features corresponding to the Golden Retriever category.

[0117] In implementation, the entity executing the model training method inputs the support image set into the first feature extraction network for feature extraction. This first feature extraction network can be an untrained original model or a pre-trained model. The support image set contains sample images of various image categories, each labeled with its category. The labeling can be represented by tags attached to the sample image or other forms. Here, the support image set is input into the basic feature extraction module of the first feature extraction network to obtain initial feature vectors output by the basic feature extraction module for each support image. These are then aggregated to obtain an initial feature vector set. This basic feature extraction module is a functional module used to extract image features from images. It typically includes at least one of convolutional layers, pooling layers, upsampling layers, and downsampling layers. The result of its feature extraction operation can be represented as a vector, a matrix, or a feature map. Here, feature vectors are used as an example; in practice, other representations can be used without affecting the final result obtained through subsequent steps. Furthermore, the aforementioned executing entity inputs the initial feature vector set output by the basic feature extraction module into the normalization processing module of the initial feature extraction model (i.e., the input of the normalization processing module is connected to the output of the basic feature processing module). This normalization processing module normalizes each initial feature vector in the initial feature vector set, thereby obtaining the normalized feature vectors output by the normalization processing module. The purpose of this normalization processing is to eliminate the performance differences between sample images from different sources after feature extraction by the basic feature extraction module, thus correcting each initial feature vector in the initial feature vector set to a smaller performance range, thereby facilitating subsequent processing.

[0118] Step S303: Use the second feature extraction network to extract features from the second image to obtain the second image features corresponding to the second image.

[0119] Here, step S303 can be implemented with reference to the specific implementation of step S112 described above.

[0120] In some implementations, the second feature extraction network can share weights with the first feature extraction network. This reduces the computational cost of the feature extraction network, facilitates subsequent feature comparison, and enables the model to generalize.

[0121] Step S304: Based on the similarity between each of the supporting features and the second image features, update each of the supporting features and the second image features to obtain updated supporting features and updated second image features.

[0122] Here, step S304 corresponds to step S103 in the foregoing embodiment, and can be implemented with reference to the specific implementation of step S103.

[0123] Step S305: Determine the loss value based on each of the updated supporting features, the updated second image features, and the category label.

[0124] Here, any suitable loss function can be used to determine the loss value, and this application embodiment is not limited in this regard. The loss function used may include, but is not limited to, at least one of the following: absolute value loss function, logarithmic loss function, squared loss function, exponential loss function, cross-entropy loss function, distance loss function, etc.

[0125] Step S306: Based on the loss value, update the model at least once to obtain the trained model.

[0126] Here, if the loss value does not meet the preset conditions, any suitable parameter update method can be used to update the network parameters of the first feature extraction network and the second feature extraction network to obtain the updated first feature extraction network and the second feature extraction network; if the loss value meets the preset conditions or the number of times the network parameters of the first feature extraction network and the second feature extraction network are updated reaches the set threshold, the current first feature extraction network model and the second feature extraction network model are determined as the trained first feature extraction network model and the second feature extraction network model.

[0127] In some implementations, the network parameters of the first feature extraction network and the network parameters of the second feature extraction network can be updated at least once based on the loss value to obtain the trained model.

[0128] When implementing, the preset conditions may include, but are not limited to, at least one of the following: the loss value is less than a set loss value threshold, the change in the loss value converges; the loss value threshold and the number of times threshold may be preset by those skilled in the art according to the actual situation, and are not limited here; the parameter update method may also be determined according to the actual situation, and may include, but are not limited to, at least one of the following: gradient descent method, Newton's momentum method, etc., and are not limited here.

[0129] In this embodiment, the model to be trained includes a first feature extraction network and a second feature extraction network. The model acquires a second image with category labels. The first feature extraction network extracts features from a set of support images to obtain a support feature set. The support image set includes at least one support image corresponding to each of multiple classification categories, and the support feature set includes support features corresponding to each classification category. The second feature extraction network extracts features from the second image to obtain second image features. Based on the similarity between each support feature and the second image feature, each support feature and the second image feature are updated to obtain updated support features and updated second image features. A loss value is determined based on the updated support features, the updated second image features, and the category labels. Based on the loss value, the network parameters of the first feature extraction network and the second feature extraction network are updated at least once to obtain the trained model. Thus, during the training of the first and second feature extraction networks, on the one hand, by updating each support feature and the second image feature based on the similarity between each support feature and the second image feature, the correlation between each support feature and the second image feature can be strengthened, deepening the understanding of the support image and the second image. Therefore, based on the similarity between the updated support features and the updated second image features, the classification accuracy of the second image can be improved. On the other hand, since the loss value used to update the network parameters of the first and second feature extraction networks is determined based on the updated support features, the updated second image features, and the category label of the second image, the ability of the trained first feature extraction network to represent support features and the ability of the second feature extraction network to represent image features can be improved, thereby enhancing the accuracy of the trained model in classifying images.

[0130] In some embodiments, determining the loss value based on each of the updated support features, the updated second image features, and the category label in step S305 above may include steps S311 to S312:

[0131] Step S311: Based on the similarity between each of the updated supporting features and the updated second image features, classify the second image to obtain the classification category of the second image.

[0132] In some implementations, the category label corresponding to the updated supporting feature with the highest feature similarity to the second image is obtained and updated, and the category label is used to determine the classification category of the second image.

[0133] In some implementations, the classification category of the second image can be determined using a classifier based on the updated second image features and the updated support features.

[0134] Step S312: Determine the loss value based on the classification category and the category label of the second image.

[0135] In implementation, based on the predicted classification category of the second image, the predicted classification category is compared with the category label of the second image to calculate the loss value. Those skilled in the art can use any suitable loss function to determine the loss value according to the actual situation, and there is no limitation here.

[0136] In some implementations, the second image is used as input to the feature extraction network, and a fully connected layer is constructed to output the predicted classification category of the second image. Based on this classification category and the category label of the second image, a cross-entropy loss is constructed and classification training is performed. Here, the cross-entropy is used as the loss function to update the parameters of the feature extraction network of the backpropagation algorithm.

[0137] In some implementations, the network parameters of the first feature extraction network, the network parameters of the second feature extraction network, and the parameters of the classifier can be updated at least once based on the loss value to obtain the trained model.

[0138] In the above embodiments, the second image is classified based on the similarity between each updated supporting feature and the updated second image features to obtain the classification category of the second image; a loss value is determined based on the classification category and the category label of the second image. Thus, during model training, the network parameters of the first and second feature extraction networks are updated based on this loss value. This improves the ability of the first feature extraction network to represent supporting features and the ability of the second feature extraction network to represent image features, making the predicted classification category more consistent with the category label of the second image, thereby improving the accuracy of image classification by the trained model.

[0139] In some embodiments, determining the loss value based on each updated supporting feature, the updated second image feature, and the category label in step S305 above may include steps S321 to S322:

[0140] Step S321: Determine the first supporting feature that corresponds to the same category as the category label, and the second supporting feature that corresponds to a different category than the category label.

[0141] In some implementations, when the category label is Miniature Pinscher, the supporting feature corresponding to the category of Miniature Pinscher is determined as the first supporting feature, and the supporting feature corresponding to the category of Miniature Pinscher is determined as the second supporting feature.

[0142] Step S322: Determine a loss value based on a first distance between the updated first support feature and the updated second image feature, and a second distance between the updated second support feature and the updated second image feature; wherein the loss value is proportional to the first distance and inversely proportional to the second distance.

[0143] Here, the first supporting feature and the second image feature can have various representation forms. Those skilled in the art can use any suitable feature representation method to represent the first supporting feature and the second image feature, and the embodiments of this application are not limited in this regard. For example, the first supporting feature and the second image feature can be represented by at least one of vectors, matrices, or feature maps.

[0144] In some implementations, the first supporting feature may include a first supporting vector, the second supporting feature may include a second supporting vector, and the second image feature may include a query vector. A first distance can be determined based on the updated first supporting vector and the updated query vector; similarly, a second distance can be determined based on the updated second supporting vector and the updated query vector. Here, the methods for calculating the first and second distances include, but are not limited to, at least one of Euclidean distance, cosine similarity, and standardized Euclidean distance.

[0145] In some implementations, after obtaining each updated support vector (including the first and second support vectors) and the query vector corresponding to the updated second image features, the loss value between the updated support vectors and the updated query vectors can be directly calculated. Based on the loss value, the network parameters of the first and second feature extraction networks are subsequently updated at least once to adjust the distance between each updated support vector and the updated query vector. For a first support vector whose corresponding classification category and the classification label of the second image are the same, the distance between the query vector and the first support vector is closer; for a second support vector whose corresponding classification category and the classification label of the second image are different, the distance between the query vector and the second support vector is farther. For example, when there is at least one second image, the loss value L can be obtained using the method shown in Formula 5 below:

[0146]

[0147] Where, N c N represents the total number of categories corresponding to each second image.Q This represents the number of second images contained in each category, where x represents the second image. This represents the backbone network and subsequent query vector updates, c. k c represents the support vectors that are consistent with the second image category. k’ represents the support vectors that are inconsistent with the second image category, d represents the cosine similarity calculation formula, and the backbone network here includes the second feature extraction network.

[0148] In the above embodiments, a first supporting feature with the same classification category and category label is determined, and a second supporting feature with a different classification category and category label is determined. Based on the first distance between the first supporting feature and the updated second image feature, and the second distance between the second supporting feature and the updated second image feature, a loss value is determined. In this way, the distance between each supporting feature and the second image feature can be adjusted and updated according to the different results of the corresponding classification category, so that the trained feature extraction network can have high model accuracy, thereby improving the accuracy of the target model for the output image recognition results.

[0149] The following describes the application of the image classification method provided in this application in a real-world scenario, using a scenario of few-sample image classification as an example.

[0150] In related technologies, few-shot classification techniques include a meta-learning-based algorithm framework. This framework receives support images and query images (corresponding to the first or second image in the aforementioned embodiments) through two branches, and then obtains the final classification result based on the feature similarity between the two. The support image, as a representative of the few-shot data, aims to serve as the central representation of each category. The query image is derived from sampling in each iteration and is also obtained from the few-shot data. However, this initial framework lacks interaction between support features and query features, resulting in unsatisfactory performance. Currently, some few-shot classification methods attempt to optimize support features by allowing interaction between them; however, the correlation between the support features and query features is not strong.

[0151] This application provides a method for training an image classification model, which can be applied to computer devices. For example... Figure 4A As shown, the method includes the following steps S401 to S405:

[0152] Step S401: Obtain input image data. The input image data is divided into two categories: support images and query image sets. The support image sets and query image sets are respectively input into the first feature extraction network and the second feature extraction network with shared weights to determine the corresponding support vectors and query vectors. Each support vector corresponds to each type of support image in the support image set, and each query vector corresponds to each query image in the query image set.

[0153] Here, the support image set includes at least one support image corresponding to each of the multiple classification categories, the support vector is the center vector representing each classification category, and the query image set is also a few-sample data, which comes from the sampling of each iteration.

[0154] Step S402: Based on the support vectors and query vectors, determine the similarity between each support vector and the query vector.

[0155] In practice, calculating the similarity between each support vector and the query vector yields a matrix, where each position indicates the similarity at (x, y), where x is a support vector and y is a query vector.

[0156] Step S403: Based on the similarity between each support vector and the query vector, update each support vector and the query vector to obtain updated support vectors and updated query vectors.

[0157] Here, we assume that the support vectors for class c are Q. c There are 32 query vectors. First, a threshold K is set to represent the number of query vectors selected. For example, K=10 means that the query vectors are sorted from highest to lowest similarity, and the top 10 most similar query vectors are selected for subsequent support vector updates. Assume the selected query vectors are T1, T2... to T... 10 Q c The similarity between these 10 vectors is A1 to A1. 10 For example, the new support vector Q can be obtained using the method shown in Formula 6 below. cnew :

[0158] Q cnew =Q c +Q n =Q c +A1*T1+A2*T2+……+A 10 *T 10 (6).

[0159] Similarly, suppose the query vector for category c is V. cThere are 10 support vectors, the number of which equals the number of categories in the image set. We first set a threshold M, where M represents the number of support vectors to select. For example, M=3 means we sort the support vectors by similarity from highest to lowest and select the top 3 most similar support vectors for subsequent query vector updates. Let's assume the selected support vectors are L1, L2, and L3. c The similarity between these three vectors is B1 to B3. For example, the new query vector V can be obtained using the method shown in Formula 7 below. cnew :

[0160] V cnew =V c +V n =V c +B1*L1+B2*L2+B3*L3 (7).

[0161] Step S404: Based on the similarity between each updated support vector and the updated query vector, classify the query image to obtain the classification category of the query image.

[0162] Here, after obtaining each updated support vector and the updated query vector, the classification category of the query image is determined by calculating the similarity between each updated support vector and the updated query vector.

[0163] Step S405: Determine the loss value based on each updated support vector and the updated query vector; update the model at least once based on the loss value to obtain the trained model.

[0164] Here, the loss value can be the loss between the predicted category of the query image and the category label of the query image, or it can be the loss between each updated support vector and the updated query vector.

[0165] Figure 4B This is a schematic diagram of the composition structure of an image classification system provided in an embodiment of this application, such as... Figure 4BAs shown, the image classification system includes a first feature extraction network 410, a second feature extraction network 420, a feature update network 430, and a classification module 440. The first feature extraction network 410 is used to determine the corresponding support vector a based on the support image I; the second feature extraction network 420 is used to determine the corresponding query vector b based on the query image H; the feature update network 430 is used to update each support vector a and query vector b based on the similarity between each support vector a and query vector b, obtaining updated support vector a and updated query vector b; the classification module 440 is used to classify the query image based on the similarity between each updated support vector a and updated query vector b, obtaining the classification category of the query image.

[0166] It should be noted that, in practice, the embodiments of this application can be applied to computer devices such as smart cameras, servers, laptops, tablets, desktop computers, smart TVs, set-top boxes, mobile phones, wearable devices, and portable gaming devices that integrate few-shot classification algorithms.

[0167] In this embodiment, each support vector and query vector are updated based on the similarity between each support vector and the query vector to obtain updated support vectors and updated query vectors. Based on the similarity between the updated support vectors and the updated query vectors, the query image is classified to obtain the category of the query image. This can strengthen the association between each support vector and the query vector, and can effectively improve the accuracy of image classification, especially in scenarios with few-sample image recognition.

[0168] Based on the foregoing embodiments, this application provides an image classification device, which includes various units and modules included in each unit. It can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0169] Figure 5A This is a schematic diagram of the composition structure of an image classification device provided in an embodiment of this application, as shown below. Figure 5A As shown, the image classification device 510 includes: an acquisition module 511, a first determination module 512, a first update module 513, and a classification module 514, wherein:

[0170] The acquisition module 511 is used to acquire the first image to be classified;

[0171] The first extraction module 512 is used to extract features from the set of support images and the first image respectively to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories;

[0172] The first update module 513 is used to update each of the supporting features and the first image features based on the similarity between each of the supporting features and the first image features, so as to obtain each of the supporting features and the first image features after the update.

[0173] The first classification module 514 is used to classify the first image based on the similarity between each of the updated supporting features and the updated first image features, so as to obtain the classification category of the first image.

[0174] In some embodiments, the first extraction module is further configured to: for each classification category, use a first feature extraction network to extract features from at least one support image in the support image set corresponding to the classification category to obtain support features corresponding to the classification category; and use a second feature extraction network to extract features from the first image to obtain the first image features.

[0175] In some embodiments, the first updating module is further configured to: update the first image feature based on the similarity between at least one of the supporting features and the first image feature to obtain the updated first image feature; and update the supporting feature for each of the supporting features based on the similarity between the supporting feature and the first image feature before the update to obtain the updated supporting feature.

[0176] In some embodiments, the first updating module is further configured to: determine at least one first target supporting feature from the supporting feature set based on the similarity between each supporting feature and the first image feature; perform a weighted summation on each first target supporting feature based on the similarity between each first target supporting feature and the first image feature to obtain a first updated feature; and update the first image feature based on the first updated feature to obtain the updated first image feature.

[0177] In some embodiments, the number of the first images is multiple; the first update module is further configured to: update the supporting features based on the similarity between the supporting features and the first image features before the update to obtain the updated supporting features, including: determining at least one target image feature based on the similarity between the first image features corresponding to each first image and the supporting features; performing a weighted summation on each target image feature based on the similarity between each target image feature and the supporting features to obtain a second updated feature; and updating the supporting features based on the second updated feature to obtain the updated supporting features.

[0178] In some embodiments, the classification module is further configured to: determine the classification category of the updated supporting feature that has the highest similarity to the first image feature based on the updated first image feature; and obtain the classification category of the first image based on the classification category of the updated supporting feature that has the highest similarity to the first image feature.

[0179] Based on the foregoing embodiments, this application provides a model training device, which includes the included units and the modules included in each unit, which can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a CPU, MPU, DSP or FPGA, etc.

[0180] Figure 5B This is a schematic diagram of the composition structure of a model training device provided in an embodiment of this application, as shown below. Figure 5B As shown, the model training device 520 includes: a second acquisition module 521, a second extraction module 522, a third extraction module 523, a second update module 524, a determination module 525, and a third update module 526, wherein:

[0181] The second acquisition module 521 is used to acquire a second image with category labels;

[0182] The second extraction module 522 is used to extract features from a set of support images using a first feature extraction network to obtain a support feature set; wherein the support image set includes at least one support image corresponding to each of multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories.

[0183] The third extraction module 523 is used to extract features from the second image using the second feature extraction network to obtain the second image features corresponding to the second image.

[0184] The second updating module 524 is used to update each of the supporting features and the second image features based on the similarity between each of the supporting features and the second image features, so as to obtain each of the supporting features and the second image features after updating.

[0185] The determination module 525 is used to determine a loss value based on each of the updated supporting features, the updated second image features, and the category label;

[0186] The third update module 526 is used to update the network parameters of the first feature extraction network and the network parameters of the second feature extraction network at least once based on the loss value, so as to obtain the trained model.

[0187] In some embodiments, the determining module is further configured to: classify the second image based on the similarity between each of the updated supporting features and the updated second image features to obtain a classification category of the second image; and determine a loss value based on the classification category of the second image and the category label.

[0188] In some embodiments, the determining module is further configured to: determine a first supporting feature whose corresponding classification category is the same as the category label, and a second supporting feature whose corresponding classification category is different from the category label; determine a loss value based on a first distance between the updated first supporting feature and the updated second image feature, and a second distance between the updated second supporting feature and the updated second image feature; wherein the loss value is proportional to the first distance and inversely proportional to the second distance.

[0189] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0190] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0191] It should be noted that, in the embodiments of this application, if the above-mentioned behavior recognition method or model training method is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0192] This application provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0193] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0194] This application provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0195] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0196] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0197] It should be noted that, Figure 6 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this application, such as... Figure 6 As shown, the hardware entities of the computer device 600 include: a processor 601, a communication interface 602, and a memory 603. The processor 601 typically controls the overall operation of the computer device 600. The communication interface 602 enables the computer device to communicate with other terminals or servers via a network. The memory 603 is configured to store instructions and applications executable by the processor 601, and can also cache data to be processed or already processed by the processor 601 and various modules in the computer device 600 (e.g., image data, audio data, voice communication data, and video communication data). It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 601, the communication interface 602, and the memory 603 can be performed via a bus 604.

[0198] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0199] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not performed. Furthermore, the coupling, direct coupling, or communication connection between the components shown or discussed may be through some interface; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0200] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this application may all be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in hardware or in a combination of hardware and software functional units.

[0201] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0202] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0203] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image classification method, characterized in that, The method includes: Obtain the first image to be classified; the number of the first images is multiple. Feature extraction is performed on the set of support images and the first image to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories; Based on the similarity between at least one of the supporting features and the first image feature, the first image feature is updated to obtain the updated first image feature; For each of the supporting features, the supporting features are updated based on the similarity between the supporting features and the first image features before the update, to obtain the updated supporting features; Based on the similarity between each of the updated supporting features and the updated first image features, the first image is classified to obtain the classification category of the first image; The step of updating the support features based on the similarity between the support features and the first image features before the update, to obtain the updated support features, includes: Based on the similarity between the first image feature corresponding to each first image and the supporting feature, at least one target image feature is determined; The weight of each target image feature is determined based on the similarity between each target image feature and the supporting feature. The target image features are then weighted and summed to obtain the second updated feature. The target image feature with a greater similarity to the supporting feature has a larger weight, and the target image feature with a smaller similarity to the supporting feature has a smaller weight. Based on the second update feature, the supporting feature is updated to obtain the updated supporting feature.

2. The method according to claim 1, characterized in that, The step of extracting features from the set of support images and the first image to obtain the support feature set and the features of the first image includes: For each of the classification categories, a first feature extraction network is used to extract features from at least one support image in the support image set corresponding to the classification category, thereby obtaining the support features corresponding to the classification category. The first image features are obtained by using a second feature extraction network.

3. The method according to claim 1, characterized in that, The step of updating the first image features based on the similarity between at least one of the supporting features and the first image features to obtain the updated first image features includes: Based on the similarity between each of the supporting features and the first image feature, at least one first target supporting feature is determined from the set of supporting features; Based on the similarity between each first target support feature and the first image feature, a weighted sum is performed on each first target support feature to obtain the first updated feature; Based on the first updated feature, the first image feature is updated to obtain the updated first image feature.

4. The method according to any one of claims 1 to 3, characterized in that, The classification of the first image based on the similarity between each updated supporting feature and the updated first image feature to obtain the classification category of the first image includes: From each of the updated support features, determine the second target support feature that has the highest similarity to the updated first image feature; The classification category corresponding to the second target supporting feature is determined as the classification category of the first image.

5. A model training method, characterized in that, The model includes a first feature extraction network and a second feature extraction network, and the method includes: Obtain a second image with category labels; Using the first feature extraction network, features are extracted from the set of support images to obtain a support feature set; wherein, the support image set includes at least one support image corresponding to each of multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories; the number of the second images is multiple. The second feature extraction network is used to extract features from the second image to obtain the second image features corresponding to the second image. Based on the similarity between at least one of the supporting features and the second image feature, the second image feature is updated to obtain the updated second image feature; For each of the supporting features, the supporting features are updated based on the similarity between the supporting features and the second image features before the update, to obtain the updated supporting features; A loss value is determined based on each of the updated supporting features, the updated second image features, and the category label; Based on the loss value, the model is updated at least once to obtain the trained model; The step of updating the support features based on the similarity between the support features and the second image features before the update, to obtain the updated support features, includes: Based on the similarity between the second image features corresponding to each second image and the supporting features, at least one target image feature is determined; The weight of each target image feature is determined based on the similarity between each target image feature and the supporting feature. The target image features are then weighted and summed to obtain the second updated feature. The target image feature with a greater similarity to the supporting feature has a larger weight, and the target image feature with a smaller similarity to the supporting feature has a smaller weight. Based on the second update feature, the supporting feature is updated to obtain the updated supporting feature.

6. The method according to claim 5, characterized in that, The step of determining the loss value based on each of the updated supporting features, the updated second image features, and the category label includes: Based on the similarity between each of the updated supporting features and the updated second image features, the second image is classified to obtain the classification category of the second image; The loss value is determined based on the classification category and the category label of the second image.

7. The method according to claim 5, characterized in that, The step of determining the loss value based on each of the updated supporting features, the updated second image features, and the category label includes: Determine a first supporting feature that corresponds to the same category as the category label, and a second supporting feature that corresponds to a different category than the category label; A loss value is determined based on a first distance between the updated first support feature and the updated second image feature, and a second distance between the updated second support feature and the updated second image feature; wherein the loss value is proportional to the first distance and inversely proportional to the second distance.

8. An image classification device, characterized in that, include: The acquisition module is used to acquire a first image to be classified; the number of the first images is multiple. The first extraction module is used to extract features from the set of support images and the first image respectively to obtain a support feature set and the first image features; wherein, the support image set includes at least one support image corresponding to each of the multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories; The first update module is configured to update the first image feature based on the similarity between at least one of the supporting features and the first image feature to obtain the updated first image feature; and for each of the supporting features, update the supporting feature based on the similarity between the supporting feature and the first image feature before the update to obtain the updated supporting feature. The first classification module is used to classify the first image based on the similarity between each of the updated supporting features and the updated first image features, so as to obtain the classification category of the first image. The first updating module is further configured to: determine at least one target image feature based on the similarity between the first image feature corresponding to each first image and the supporting feature; determine the weight of each target image feature based on the similarity between each target image feature and the supporting feature; perform a weighted summation on each target image feature to obtain a second updated feature; wherein the target image feature with a greater similarity to the supporting feature has a greater weight, and the target image feature with a smaller similarity to the supporting feature has a smaller weight; and update the supporting feature based on the second updated feature to obtain the updated supporting feature.

9. A model training device, characterized in that, include: The second acquisition module is used to acquire a second image with category labels; the number of the second images is multiple. The second extraction module is used to extract features from a set of support images using a first feature extraction network to obtain a support feature set; wherein the support image set includes at least one support image corresponding to each of multiple classification categories, and the support feature set includes support features corresponding to each of the classification categories. The third extraction module is used to extract features from the second image using the second feature extraction network to obtain the second image features corresponding to the second image. The second update module is configured to update the second image feature based on the similarity between at least one of the supporting features and the second image feature to obtain the updated second image feature; and for each of the supporting features, update the supporting feature based on the similarity between the supporting feature and the second image feature before the update to obtain the updated supporting feature. The determination module is used to determine a loss value based on each of the updated supporting features, the updated second image features, and the category label; The third update module is used to update the model at least once based on the loss value to obtain the trained model. The second updating module is further configured to: determine at least one target image feature based on the similarity between the second image feature corresponding to each second image and the supporting feature; determine the weight of each target image feature based on the similarity between each target image feature and the supporting feature; perform a weighted summation on each target image feature to obtain a second updated feature; wherein the target image feature with a greater similarity to the supporting feature has a greater weight, and the target image feature with a smaller similarity to the supporting feature has a smaller weight; and update the supporting feature based on the second updated feature to obtain the updated supporting feature.

10. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.

Images