Model training method, image classification method and related equipment

Abstract

The invention relates to artificial intelligence, and provides a model training method, an image classification method and related equipment. The method comprises the following steps: acquiring a first feature vector of a first image through an attention head in a first model, and acquiring a second feature vector corresponding to the first feature vector through a multi-layer sensor of the first model; based on the weight of the attention head in the first model and the weight of the multi-layer perceptron of the first model, fusing the first feature vector and the second feature vector to obtain a first fusion feature; classifying the first image by using a first model according to the first fusion feature to obtain a first prediction category; determining a first loss value of the first model according to the first prediction category, and training the first model based on the first loss value to obtain a second model; a first score of the attention head in the second model is determined based on the second image, and a weight of the attention head in the second model is updated based on the first score. According to the method, the image classification accuracy of the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a model training method, an image classification method, and related equipment. Background Art

[0002] In recent years, the development of generative artificial intelligence has led to an explosive growth in the amount of data it generates, making the issue of determining whether data is generated by artificial intelligence a hot topic. The classification methods used in related technologies mainly include two categories: the first category is score-based methods, which mainly obtain distinguishing features through methods such as image semantic inconsistency and statistical artifacts for judgment, but this method usually relies on a specific generator, and the distinguishing features are prone to failure due to generator transformation, making it difficult to accurately classify data. The second method uses supervised classification. Although it does not rely on a specific generator and can process multiple data sources, it performs poorly when facing unseen fields and new generators, has weak generalization capabilities, and has low accuracy in detection results in new scenarios. Summary of the Invention

[0003] This application provides a model training method, an image classification method, and related equipment to solve the problem in related technologies of difficulty in accurately identifying whether data is generated by artificial intelligence due to reliance on specific generators, insufficient model generalization capabilities, and other reasons.

[0004] A first aspect of an embodiment of the present application provides a model training method, the method comprising: obtaining a first eigenvector of a first image through an attention head in a first model, and obtaining a second eigenvector corresponding to the first eigenvector through a multilayer perceptron of the first model; fusing the first eigenvector and the second eigenvector based on the weights of the attention head in the first model and the weights of the multilayer perceptron of the first model to obtain a first fused feature; classifying the first image using the first model according to the first fused feature to obtain a first predicted category; determining a first loss value of the first model according to the first predicted category, training the first model based on the first loss value to obtain a second model; determining a first score of the attention head in the second model based on the second image, and updating the weight of the attention head in the second model based on the first score.

[0005] A second aspect of an embodiment of the present application provides an image classification method, which includes: using an image classification model to obtain an image category corresponding to an input image, wherein the image classification model is obtained by executing the model training method of the first aspect mentioned above.

[0006] A third aspect of an embodiment of the present application provides a chip system, which is applied to a computer device. The chip system includes one or more processors, which are used to call computer instructions to enable the computer device to input training data into the chip system and execute the steps in the method provided in the first or second aspect above.

[0007] The fourth aspect of an embodiment of the present application provides a model training device, which includes: an acquisition unit, used to acquire a first eigenvector of a first image through an attention head in a first model, and to acquire a second eigenvector corresponding to the first eigenvector through a multilayer perceptron of the first model; a fusion unit, used to fuse the first eigenvector and the second eigenvector based on the weights of the attention head in the first model and the weights of the multilayer perceptron of the first model to obtain a first fused feature; a classification unit, used to classify the first image using the first model according to the first fused feature to obtain a first predicted category; a training unit, used to determine a first loss value of the first model according to the first predicted category, and train the first model based on the first loss value to obtain a second model; an updating unit, used to determine a first score of the attention head in the second model based on the second image, and update the weight of the attention head in the second model based on the first score.

[0008] A fifth aspect of an embodiment of the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method provided in the first or second aspect above when executing the computer program.

[0009] A sixth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the first or second aspect are implemented.

[0010] A seventh aspect of the embodiments of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps in the method provided in the first or second aspect above.

[0011] In the model training method of this embodiment, the first image is processed using the attention heads in the first model to extract a first eigenvector that reflects key information about the image. The first eigenvector is further processed using the multilayer perceptron in the model to obtain a corresponding second eigenvector. When training an image classification model, a multi-level feature extraction strategy can be used to extract low-dimensional detail features of the first image, which serves as training data, to obtain a first eigenvector. Feature extraction is then performed on the first eigenvector to obtain a high-dimensional second eigenvector. By comprehensively considering the weights of the attention heads in the first model and the weights of the multilayer perceptron, the first eigenvector is fused with the second eigenvector to obtain a first fused feature. This first fused feature can enhance the feature expression capability of the first image, enabling the model to more accurately learn the features of the first image. Based on this fused feature, the first model is used to classify the first image, thereby obtaining a first predicted category. Based on the difference between the first predicted category and the true category, a first loss value for the first model is determined, and the first model is trained based on this loss value. After training, a second model is obtained, completing the initial training of the model. Based on the second image, a first score for the second model's encoder attention head is determined, and the weights of the second model's encoder attention head are updated based on this score to optimize the model's performance and feature extraction capabilities. After initially training the first model to obtain the second model, the second model's performance can be evaluated using the second image in the independent validation set. By optimizing the attention head weights based on the evaluation results, the model can be optimized more specifically, thereby improving the model's prediction accuracy and generalization capabilities for image classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a flowchart of a pre-training method for a model provided in one embodiment of the present application; Figure 2 This is an example diagram of the network structure of the model provided in one embodiment of the present application; Figure 3 This is an example diagram of the network structure of a model provided by another embodiment of the present application; Figure 4 is a flowchart of a method for determining a second score provided in one embodiment of the present application; Figure 5 This is an example diagram of the network structure of a model provided by another embodiment of the present application; Figure 6 This is a flowchart of a method for training a residual feature classifier provided in one embodiment of the present application; Figure 7 This is a schematic diagram illustrating the principle of a residual feature classifier provided in one embodiment of the present application; Figure 8 This is a flowchart of a method for obtaining an image category using an image classification model provided by an embodiment of the present application; Figure 9 This is a functional module diagram of a model training device provided in one embodiment of the present application; Figure 10 It is a structural diagram of a computer device for implementing a model training method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features therein can be combined with each other in the absence of conflict.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing embodiments in some embodiments only and are not intended to limit this application.

[0016] It should be noted that, in this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A alone, A and B together, and B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," and so on (if any) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or precedence.

[0017] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner. The following embodiments and features in the embodiments may be combined with each other unless there is a conflict.

[0018] In some cases, generative AI (AI) technology has made breakthroughs in recent years. Generative models, such as diffusion models, generative adversarial networks, and large language models, have demonstrated powerful content production capabilities. These models use deep neural networks to learn the distribution patterns of massive amounts of data, enabling them to generate highly realistic and diverse content such as artificial intelligence generated images (AIGI).

[0019] The explosive growth of generated content has triggered a new crisis in the field of AI security. On the one hand, malicious actors are exploiting generative models to rapidly create disinformation, deepfakes, and phishing materials, severely undermining the security of the information ecosystem. On the other hand, the distinction between AI-generated content and human creations is becoming increasingly blurred in areas such as copyright. Against this backdrop, artificial intelligence-generated content detection (AI-GCD) has become a critical technology that urgently needs to be addressed. Its core goal is to develop efficient algorithms that can accurately distinguish whether input content is AI-generated.

[0020] The methods used in related technologies mainly fall into two categories: the first category is feature score-based detection methods, which make judgments by extracting distinguishing features between generated content and human creations based on statistical or physical characteristics. For example, in the image field, the distinguishing features extracted by this method can be semantic inconsistency or generated fingerprint inconsistency. In the text field, the distinguishing features extracted by this method can be statistical artifacts, model perplexity, intrinsic dimensional stability, etc. The drawback of this type of method is that it strongly relies on a specific generator architecture and training data distribution, and the features are easily destroyed by adversarial attacks (such as image compression and text interpretation).

[0021] The second category is supervised classification methods, which train binary classification models by constructing labeled datasets (including both human-created and AI-generated samples). Although this method does not rely on prior knowledge and is adaptable to multimodal data, its generalization performance is severely limited by the domain coverage of the training data. When using unseen generators or content after style transfer, the classifier often suffers from a sharp drop in accuracy due to capturing spurious correlations in the dataset (such as specific watermarks or compression algorithms). Furthermore, the cost of generating adversarial examples is far lower than the cost of training the detection model, further exacerbating the asymmetry between attack and defense.

[0022] Therefore, it is difficult for relevant technologies to balance detection accuracy and generalization ability in AIGI detection. Especially when facing the continuously evolving generative models, the timeliness, robustness and cross-domain adaptability of the detection system have become technical bottlenecks that need to be broken through urgently.

[0023] In order to solve the above problems, the embodiment of the present application provides a model training method that can use a multi-level feature extraction strategy when training an image classification model to simultaneously capture the low-dimensional detail features and high-dimensional semantic features of the image, and enhance the feature expression capability through an adaptive fusion module, so that the model can more accurately capture the statistical artifacts and inconsistencies unique to the generated content. Secondly, a dynamic feedback training mechanism is introduced. After the initial classification task training, an independent validation set is used to evaluate the model performance. The model parameters are optimized based on the evaluation results to improve the model's generalization ability for new generators. It can improve the recognition accuracy of AIGI and effectively solve the problems of poor adaptability and insufficient robustness in the field of traditional detectors.

[0024] The model training method provided in the embodiments of the present application can be executed by a computer device, specifically by a processor of the computer device. The computer device may include a terminal device or a server. The terminal device may be a mobile phone, a tablet computer, a desktop computer, a portable notebook, etc. The server may be an independent server, a server cluster consisting of multiple servers, or a cloud server capable of cloud computing.

[0025] refer to Figure 1 As shown, it is a flowchart of the model training method provided in one embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0026] S101, obtaining a first eigenvector of a first image through an attention head in a first model, and obtaining a second eigenvector corresponding to the first eigenvector through a multilayer perceptron of the first model.

[0027] In some embodiments, in order to train the first model, a training set can be first obtained. For example, the training set can be obtained by receiving user input or obtaining it from an open source database. The training set may include m images, where m represents a preset integer and can be set according to actual needs. The m images may include Artificial Intelligence Generated Image (AIGI) images and non-AIGI images, and each image may have a corresponding category label, for example, the category label may be real (Real) or fake (Fake). Among them, non-AIGI images correspond to real labels, and AIGI images correspond to fake labels. For example, 0 can be set to represent real labels and 1 can be set to represent fake labels. Any image in the training set can be used as the first image, for example, the cth image among the m images can be used as the cth first image, and c represents an integer greater than or equal to 1 and less than or equal to m.

[0028] In some embodiments, the first model may include an encoder including a plurality of attention heads. Figure 2 As shown in FIG, a diagram illustrating a network structure example of the first model provided in an embodiment of the present application. The first model may be a Vision Transformer (ViT) model, and the encoder may be a Transformer Encoder. For example Figure 3 As shown, the Transformer encoder layer contains a Multi-Head Self-Attention (MHSA) module, which consists of k attention heads { }, where k represents a preset integer and can be set according to actual needs. s represents an integer greater than or equal to 1 and less than or equal to k, represents the sth attention head.

[0029] In some embodiments, when obtaining the first feature vector of the first image using the attention head in the first model, the first image can be segmented into image blocks to obtain image block vectors corresponding to the image blocks. Linear mapping and position encoding are performed on the image block vectors to obtain a first embedding vector. The first embedding vector is then calculated using the attention head in the first model to obtain the first feature vector.

[0030] For example Figure 2As shown in the figure, when the ViT model obtains the first feature vector of the first image, it can divide the input first image into multiple image patches. The multiple image patches can be blocks of the same size. For example, a first image of size H×W×C can be evenly divided into N² image patches of size (H / N)×(W / N)×C, where H represents the length of the first image, W represents the width of the first image, C represents the number of channels of the first image, and N represents a preset integer that can be set according to actual needs.

[0031] When obtaining the image block vector corresponding to the image block, the ViT model can flatten each image block into a vector to obtain the image block vector. For example, an image block of (H / N)×(W / N)×C=16×16×3 can be flattened into a 768-dimensional vector.

[0032] When linearly projecting image block vectors and adding positional encodings, the ViT model flattens and maps each image block into a D-dimensional linear space using a learnable linear transformation matrix, generating an initial embedding for each image block. D represents a preset integer that can be set as needed. A learnable positional encoding vector is then added to each initial embedding to generate a first embedding containing spatial position information.

[0033] refer to Figure 2 As shown in FIG, when the first feature vector is obtained by calculating the first embedding vector through the attention head, all the first embedding vectors corresponding to all image blocks can be used to form an input vector sequence, which is input into a deep network stacked by L transformation encoder layers, where L represents a preset integer and can be set according to actual needs, for example, L=1. Figure 3 As shown, the input vector sequence can be passed to the k attention heads of the encoder at the same time, and the k attention heads perform scaled dot product attention parallel calculation in parallel to obtain the first feature vector.

[0034] Based on the above embodiment, each attention head calculates the correlation (e.g., attention weight) between each first embedding vector and all other first embedding vectors, thereby allowing the model to simultaneously focus on semantic information at different positions of the image and obtain a new feature representation, such as the first feature vector.

[0035] In one example, the sth attention head The formula used can be exemplified as: , in, is the query matrix obtained after linear transformation of the input vector sequence, express The submatrix corresponding to the sth attention head after splitting along the feature dimension. K is the key matrix after the input vector sequence is linearly transformed, represents the submatrix corresponding to the s-th head after K is split along the feature dimension, express The transposed matrix of is the key dimension, which represents the length of each key vector in the key matrix K. V is the value matrix after the linear transformation of the input vector sequence, Represents the submatrix corresponding to the s-th head after V is split along the feature dimension. express function, Represents the first eigenvector output by the sth attention head.

[0036] In some embodiments, the first model may include a Multi-Layer Perceptron (MLP), for example Figure 3 As shown in the figure, MLP is located at the last part of the Transformer encoder layer, which receives the first eigenvector output by MHSA and performs nonlinear transformation on the first eigenvector to obtain the second eigenvector.

[0037] In one example, the MLP may include a first fully connected layer, which is used to expand the dimension of the first feature vector to a higher dimension to enhance its feature representation capability. The MLP also includes a nonlinear activation function (e.g., a GELU function) for performing a linear transformation on the vector output by the first fully connected layer to obtain a nonlinear transformation result. The MLP also includes a second fully connected layer, which is used to compress the dimension of the nonlinear transformation result to the original dimension, e.g., the dimension of the first feature vector, to obtain a second feature vector.

[0038] In one example, the MLP may include Network layers, Indicates a preset integer that can be set according to actual needs. For example Figure 3 As shown, the kth attention head The first eigenvector of the output Input MLP, the second eigenvector of MLP output can be expressed as ,in, represents the first The second eigenvector output by the network layer is Indicates greater than or equal to 1 and less than or equal to An integer.

[0039] Based on the above embodiment, a multi-level feature extraction strategy can be used when training a model to obtain low-dimensional detail features of the first image as training data to obtain a first feature vector, and further perform feature extraction on the first feature vector to obtain a high-dimensional second feature vector.

[0040] S102: Based on the weights of the attention heads in the first model and the weights of the multilayer perceptron of the first model, the first eigenvector and the second eigenvector are fused to obtain a first fused feature.

[0041] In some embodiments, when determining the first fusion feature, the weight of the attention head in the first model can be used as the weight of the corresponding first eigenvector; the weight of the multilayer perceptron of the first model can be used as the weight of the corresponding second eigenvector. According to the weight of the first eigenvector and the weight of the second eigenvector, the first eigenvector and the second eigenvector are weighted and summed, and the first fusion feature is obtained according to the result of the weighted summation. Among them, the weight of the attention head and the weight of the network layer are preset hyperparameters, which can be set according to actual needs, or obtained through learning when training the first model. For example, the value range of the weight is in the interval [0,1].

[0042] In one example, for the kth attention head, the formula used to calculate the weighted fusion result can be exemplarily expressed as: ,in, represents the first fusion feature corresponding to the cth first image, represents the kth attention head The first eigenvector of the output, represents the kth attention head The weight of . represents the first The second eigenvector output by the network layer is represents the first The weights of the network layers, Indicates the number of network layers of MLP.

[0043] Based on the above embodiment, the weights of the attention heads of the first model and the weights of the multilayer perceptron are comprehensively considered, and the first eigenvector and the second eigenvector are fused to obtain a first fused feature. The feature expression ability of the first image can be enhanced based on the first fused feature, so that the model can learn the features of the first image more accurately.

[0044] S103: Classify the first image using a first model according to the first fusion feature to obtain a first predicted category.

[0045] In some embodiments, the first model includes a classifier, for example, a binary classifier based on a logistic regression model. The classifier can be used to classify the first fused features to determine a first predicted category for the first image. For example, the result of the first predicted category can be true or false.

[0046] In one example, the formula used by the classifier to determine the first predicted category can be exemplarily expressed as: ,in, represents the sigmoid function, represents the weight vector of the classifier, represents the first fusion feature, Indicates the preset bias term. represents the predicted probability, if Closer to 1, it can indicate that the first predicted category is false; if A value closer to 0 may indicate that the first predicted category is true.

[0047] S104: Determine a first loss value of the first model according to the first prediction category, and train the first model based on the first loss value to obtain a second model.

[0048] In some embodiments, when determining the first loss value based on the first prediction category, the loss function that can be used includes, but is not limited to, cross-entropy loss (Cross-Entropy Loss), mean squared error, etc. When training the first model based on the first loss value, the gradient of the loss function with respect to the model parameters can be calculated, and based on the gradient, the model parameters can be updated using a preset optimization algorithm and learning rate until the model meets preset requirements. For example, the optimization algorithm may include, but is not limited to, Adam, etc., and the model parameters may include, but are not limited to, the weight parameters of the first model, such as the weights of the attention head, the weights of the network layer, the bias, etc. The preset requirements may include, but are not limited to, the convergence of the loss value, the number of iterations reaching a preset threshold, etc.

[0049] S105 , determining a first score of the attention head in the second model based on the second image, and updating the weight of the attention head in the second model based on the first score.

[0050] In some embodiments, the second image represents a validation image in a validation set, for example, the validation image may be an AIGI image or a non-AIGI image with a class label. The validation set may be a dataset independent of the training set, used to evaluate the generalization ability of the model.

[0051] In some embodiments, when determining the first score of the attention head in the second model based on the second image, the second prediction category of the second image can be determined based on the attention head in the second model, and the first score of the attention head of the encoder in the second model can be determined based on the accuracy and recall rate of the second prediction category.

[0052] In one example, for ease of description, the attention head in the second model is referred to as the attention head. When determining the second prediction category of the second image, the third eigenvector corresponding to the second image can be obtained through the attention head in the second model. The third eigenvector is nonlinearly transformed through the multilayer perceptron of the second model to obtain a fourth eigenvector. According to the fourth eigenvector, the second image is classified through the second model to obtain the second prediction category. For example, a fusion vector of the third eigenvector and the fourth eigenvector is obtained, and the fusion vector is classified through the classifier of the second model to obtain the second prediction category. Based on the second prediction category and the preset category label of the second image, the accuracy and recall rate of the second prediction category are determined.

[0053] In one example, for each attention head, the precision and recall of the second predicted class can be determined. Precision represents the percentage of correctly predicted classes, and recall represents the proportion of true positive samples that are correctly predicted. A true positive sample can be a second image with a true label (e.g., a non-AIGI image). The first score for each attention head can be determined based on the sum or weighted sum of the precision and recall of the second predicted class.

[0054] In one example, if the trained model has 8 attention heads, the validation set contains 500 AIGI images and 500 non-AIGI images. For each validation image, 8 sets of features are generated using the 8 attention heads. Each set of features is combined with the MLP features to form a fused feature. The fused features are then classified using LR, resulting in a total of 8 second-predicted categories corresponding to the 8 attention heads. The precision of the second-predicted category is calculated as: number of correct predictions / total number of samples (1000); the recall of the second-predicted category is calculated as: number of correctly predicted positive samples / number of true positive samples (500). The first score of each attention head is calculated as 0.5 × precision + 0.5 × recall.

[0055] Based on the above embodiment, the contribution of each attention head can be independently evaluated using the first score, thereby more accurately understanding the internal working principles of the model and improving the interpretability of the model. The larger the first score, the greater the contribution of the corresponding attention head to obtaining the correct prediction result.

[0056] In some embodiments, when updating the model parameters of the trained model based on the first score, a first score for each attention head in the second model may be determined, and an attention head may be selected from all attention heads in the second model based on the maximum value of the first scores. The weight of the selected attention head is updated such that the updated weight of the selected attention head is greater than the weight of the unselected attention heads.

[0057] In one example, a preset number of attention heads with the largest first scores can be selected based on the first scores, and the weights of the selected attention heads can be updated to larger values, while the weights of the other unselected attention heads remain unchanged, so that the updated weights of the selected attention heads are greater than the weights of the unselected attention heads. The preset number can be set based on actual needs and is not specifically limited in this application.

[0058] In one example, the training process of the above embodiment can be repeated on different training and validation sets to obtain a final optimal set of attention heads, for example, a set consisting of a preset number of attention heads with the highest first scores. Weights can be assigned to the optimal attention heads based on their frequency of occurrence across different training and validation sets.

[0059] Based on the above embodiment, after the first model is initially trained to obtain the second model, the model performance of the second model can be evaluated using the second image in the independent validation set. By optimizing the weights of the attention heads through the evaluation results, the model can be optimized more specifically, thereby improving the model's prediction accuracy and generalization ability for image classification tasks.

[0060] In some embodiments, in addition to the method described above for optimizing the weights of attention heads to improve the model's prediction accuracy and generalization capabilities, a search for important features can be performed to identify the feature components that have the least impact on the model's prediction results. During inference, the weights of these feature components with the least impact on the model's prediction results can be reduced, thereby improving the accuracy of the model's prediction results. The model can be the first model or the second model, or a model (e.g., a third model) obtained by updating the parameters of the second model based on the first score.

[0061] In some embodiments, when searching for important features, a fifth eigenvector of the third image can be obtained through an attention head; a second score of a feature component of the fifth eigenvector is determined, and a weight of the corresponding feature component is updated based on the second score. The attention head can be the attention head in the first model, the attention head in the second model, or the attention head in the third model.

[0062] In one example, the third image may be an image in the validation set. When obtaining the fifth eigenvector of the third image using the attention head, the third image may be segmented into image blocks, and image block vectors corresponding to the image blocks may be obtained. Linear mapping and position encoding are performed on the image block vectors to obtain corresponding embedding vectors. The fifth eigenvector is then calculated using the attention head on this embedding vector.

[0063] In one example, the method for determining the second score of the characteristic component of the fifth characteristic vector can refer to Figure 4 As shown in the flow chart. Figure 4 As shown, it is a flowchart of the method for determining the second score provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0064] S401 , performing a nonlinear transformation on the fifth eigenvector through a multilayer perceptron to obtain a corresponding sixth eigenvector.

[0065] In some embodiments, the multilayer perceptron may be a multilayer perceptron of the model to which the attention head belongs. For example, if the attention head is an attention head in a first model, the multilayer perceptron may be a multilayer perceptron of the first model. For another example, if the attention head is an attention head in a third model, the multilayer perceptron may be a multilayer perceptron of the third model.

[0066] In some embodiments, the third eigenvector can be transformed nonlinearly by a multilayer perceptron to obtain the sixth eigenvector. In one example, the sth attention head of the ViT model Get the fifth eigenvector corresponding to the third image . refer to Figure 5 As shown, the network layer of the MLP of the ViT model receives the fifth feature vector And output the corresponding sixth eigenvector through the last layer of network Thus, for the k fifth eigenvectors { }, we can get k corresponding sixth eigenvectors { }.

[0067] S402 : Remove the feature components of the first dimension in the sixth eigenvector to obtain an updated sixth eigenvector.

[0068] In some embodiments, the fifth feature vector obtained for the s-th attention head is , the feature components of the first dimension can be removed. The first dimension can be any dimension, such as the i-th dimension, where i represents a preset integer. Remove the fifth eigenvector The characteristic component of the i-th dimension in , the updated fifth eigenvector is obtained .

[0069] In some embodiments, the fifth feature vector obtained for the s-th attention head is , and its corresponding sixth eigenvector is The sixth eigenvector can be removed The characteristic component of the first dimension in , for example Figure 5 As shown, the sixth eigenvector The feature component of the first dimension can be the feature component of the i-th dimension , get the updated sixth eigenvector For k attention heads, we can get k updated sixth eigenvectors { }.

[0070] S403 : Based on the updated sixth eigenvector, classify the third image using the classifier of the model to obtain a third predicted category.

[0071] In some embodiments, a third predicted category is determined based on the updated sixth feature vector by a classifier of the model, wherein the classifier may be a trained LR model.

[0072] S404 : Determine a score of the classifier according to the third predicted category, and determine a second score of the feature component of the first dimension based on the score of the classifier.

[0073] In some embodiments, a classifier score (e.g., precision, recall, etc.) on a validation set can be determined based on the third predicted category, and this score can be used as the second score. Since the validation set, training set, and test set are from different domains, the third score can be referred to as an out-of-domain score. Thus, for each dimension's feature component, a corresponding second score can be obtained using the methods described in the above embodiments.

[0074] In some embodiments, a preset number of feature components of the out-of-domain second score can be recorded, where the preset number can be a preset proportion of the number of feature dimensions, such as 30%. These feature components can be considered to have the least impact on the prediction results.

[0075] In some embodiments, when updating the weight of the corresponding feature component based on the second score, if the second score is less than a preset score threshold, the weight of the feature component of the first dimension is updated to a smaller value, such that the updated weight of the feature component of the first dimension is less than the weights of the feature components of other dimensions. The preset score threshold can be set according to actual needs and is not specifically limited in this application.

[0076] In some embodiments, the training set and validation set can be swapped, and the above-described feature removal, validation, and recording processes can be repeated. This process is terminated when the second score no longer improves. Through the above process, the feature components that have the least impact on the prediction results can be found. By reducing the weights of these feature components during inference, the importance of feature components can be selected. In one example, after updating the weights of the feature components of the corresponding dimensions based on the second score, the model can be trained based on the updated weights of the feature components.

[0077] In one example, if the sixth eigenvector has three eigencomponents (hereinafter referred to as components), and all components participate in the classification with the same weight of 1, the baseline probability of a correct classification is 0.8. Each component is removed individually to observe the change in the probability of correct classification: after removing the first component, the probability of correct classification increases to 0.9; after removing the second component, the probability of correct classification decreases to 0.6; after removing the third component, the probability of correct classification is 0.95. For each component, the change in probability after removal is calculated: for the first component, 0.9 - 0.8 = 0.1; for the second component, 0.6 - 0.8 = -0.2; for the third component, 0.95 - 0.8 = 0.15. The component with the largest decrease in probability is identified, for example, the component with the largest negative change and the largest absolute value: the second component, with a change of -0.2. The sum of the absolute values ​​of all changes is calculated: 0.1 + 0.2 + 0.15 = 0.45. The absolute value of the change in the component with the largest decrease in probability, 0.2, is used as the numerator, and the sum of the absolute values ​​of the changes, 0.45, is the denominator. For the second component, the weight adjustment factor is calculated as: 1-(0.2 / 0.45)=0.9556. Therefore, the weight of the second component can be adjusted down to 0.9556. The weights of other components (such as the first and third components) remain at 1 because their removal does not significantly reduce classification performance, but rather improves it. In this way, the weights of components that have a negative impact on classification performance can be reduced, while the weights of components that have a positive impact (or at least no negative impact) on classification performance can be maintained or possibly increased.

[0078] Based on the above embodiment, the importance of features is evaluated by removing feature components and observing the change in scores on the validation set. Specifically, by training the LR model - removing features - validating - recording low-scoring features - exchanging datasets and repeating - reducing the weights of unimportant features, the weights of feature components that have the least impact on the prediction results can be evaluated and reduced, thereby improving the model's generalization ability and prediction accuracy.

[0079] In some embodiments, the model's classifier can be further trained. By training the residual feature classifier on large-scale image data, the feature differences between non-AIGI and AIGI in the embedding space can be learned, thereby improving the accuracy of the model's image classification results.

[0080] In one example, the method for training a residual feature classifier can be referred to Figure 6 As shown in the flow chart. Figure 6 As shown, it is a flowchart of the method for training a residual feature classifier provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0081] S601: Obtain a residual vector between the embedding vector of the fourth image and the embedding vector of the fifth image.

[0082] In some embodiments, the embedding vector of the fourth image (e.g., the second embedding vector) and the embedding vector of the fifth image (e.g., the third embedding vector) can be obtained using a pre-trained multimodal data processing model. For example, the multimodal data processing model may include a CLIP (Contrastive Language–Image Pre-training) model. The CLIP model can embed images and text into the same high-dimensional space, so that similar images and text are close to each other in the embedding space.

[0083] In some embodiments, the fourth image may be a non-AIGI image in a preset non-AIGI image library, such as a real image (Real Image). The fifth image may be an AIGI image in the AIGI image library, such as a fake image (FakeImage). Figure 7 As shown, the fourth image and the fifth image are input into the CLIP model, and the CLIP image encoder generates an embedding vector for the fourth image and an embedding vector for the fifth image. In one example, the CLIP image encoder can also be the encoder of the ViT model in the above embodiment.

[0084] In some embodiments, a residual vector may be calculated between the second embedding vector and the third embedding vector, which may indicate the difference between the non-AIGI image and the AIGI image. Figure 7 As shown, a subtraction operation can be performed between the second embedding vector and the third embedding vector to obtain a residual vector, and the residual vector can also be normalized.

[0085] S602: Predict the category of the fifth image using a classifier according to the residual vector to obtain a third predicted category.

[0086] In some embodiments, reference Figure 7As shown, the normalized residual vector is fed into the LR model of the classifier for classification to obtain a third predicted category. The third predicted category may be the predicted category corresponding to the fifth image, such as real or fake.

[0087] S603: Determine a second loss value of the classifier according to the third predicted category, and train the classifier based on the second loss value.

[0088] In some embodiments, the second loss value is determined based on the third prediction category and a preset loss function. In one example, the preset loss function includes but is not limited to cross entropy loss, and the formula used can be exemplarily expressed as: .in, represents the second loss value, and n represents the number of training data, such as the number of fifth images. is the category label of the rth fifth image, for example, Indicates that the category label of the rth fifth image is false, Indicates that the category label of the rth fifth image is true. is the predicted probability of the rth fifth image, if Closer to 1, it can indicate that the predicted category is false; if A value closer to 0 may indicate that the first predicted category is true.

[0089] In some embodiments, by minimizing the second loss value, the classifier can learn the feature differences of AIGI images relative to non-AIGI images in the CLIP embedding space, thereby improving the classification accuracy of the classifier for images.

[0090] In some embodiments, a residual vector library and a non-AIGI image feature library can be constructed to achieve fast and accurate image classification in the model inference stage.

[0091] In one example, the residual vectors obtained for each pair of non-AIGI images (e.g., the fourth image) and AIGI images (e.g., the fifth image) are recorded to construct a residual vector library. The residual vector library contains all residual vectors used for training, which represent the feature differences between non-AIGI images and AIGI images in the CLIP embedding space.

[0092] In one example, the non-AIGI image embedding vector corresponding to each residual vector (e.g., the embedding vector of the fourth image) is recorded to construct a non-AIGI image feature library. The non-AIGI image feature library contains the embedding vectors of all non-AIGI images used for training. These embedding vectors will be used to calculate the residual vector between the input image and the non-AIGI image during the inference phase.

[0093] In one example, during the model inference phase, when a new image is input, the CLIP model can first be used to extract its feature vector. This feature vector is then compared with vectors in the non-AIGI image feature library to calculate a residual vector. Finally, this residual vector is compared with vectors in the residual vector library for similarity to determine whether the input image is real or fake. For example, the similarity can be cosine similarity. If the similarity is greater than a preset threshold T (e.g., 0.6), the input image can be determined to be an AIGI image. If the similarity is less than or equal to the preset threshold T (e.g., 0.6), the input image can be determined to be a non-AIGI image.

[0094] Based on the above examples, the residual feature classifier can learn the characteristic differences between real and fake images in the CLIP embedding space on large-scale image data. By calculating residual vectors and training the classifier, the model can accurately identify and classify images, effectively improving image classification accuracy. During the model inference phase, the constructed residual vector library and non-AIGI image feature library can quickly and accurately determine the authenticity of input images, providing strong support for image authentication.

[0095] The model training method provided in the embodiment of the present application processes the first image through the attention head in the first model to extract a first eigenvector that can reflect the key information of the image; and further processes the first eigenvector using the multi-layer perceptron in the model to obtain the corresponding second eigenvector. When training the image classification model, a multi-level feature extraction strategy can be used to obtain the low-dimensional detail features of the first image as training data to obtain the first eigenvector, and further feature extraction is performed on the first eigenvector to obtain the high-dimensional second eigenvector. Taking into account the weights of the attention head of the first model and the weights of the multi-layer perceptron, the first eigenvector is fused with the second eigenvector to obtain a first fused feature. The feature expression ability of the first image can be enhanced based on the first fused feature, so that the model can learn the features of the first image more accurately. Based on this fused feature, the first model is used to classify the first image to obtain a first predicted category. According to the difference between the first predicted category and the true category, the first loss value of the first model is determined, and the first model is trained based on the loss value. After training, the second model is obtained to achieve initial training of the model. Based on the second image, a first score for the second model's encoder attention head is determined, and the weights of the second model's encoder attention head are updated based on this score to optimize the model's performance and feature extraction capabilities. After initially training the first model to obtain the second model, the second model's performance can be evaluated using the second image in the independent validation set. By optimizing the attention head weights based on the evaluation results, the model can be optimized more specifically, thereby improving the model's prediction accuracy and generalization capabilities for image classification tasks.

[0096] The above embodiment describes how to train a model, and the trained model can be used as an image classification model to perform image classification tasks. An image classification method provided in one embodiment of the present application includes: using an image classification model to obtain an image category corresponding to an input image. The image classification model is obtained by executing the above model training method.

[0097] In one example, the method of using the image classification model to obtain the image category can be referred to Figure 8 As shown in the flow chart. Figure 8 As shown, it is a flowchart of a method for obtaining image categories using an image classification model provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0098] S801, using an image classification model to segment an input image into image blocks, and obtaining image block vectors corresponding to the image blocks.

[0099] S802: Perform linear mapping on the image block vector to obtain an embedding vector.

[0100] S803: Calculate the embedded vector to obtain a first eigenvector.

[0101] In some embodiments, the specific implementation of steps S801 to S803 can refer to the embodiment of step S101 and will not be described again.

[0102] S804: Perform a nonlinear transformation on the first eigenvector to obtain a second eigenvector, and fuse the first eigenvector and the second eigenvector to obtain a fused feature.

[0103] In some embodiments, the specific implementation of step S804 can refer to the embodiment of step S102 and will not be described again.

[0104] S805: Classify the input image based on the fused features to obtain an image category.

[0105] In some embodiments, the specific implementation of step S805 can refer to the embodiment of step S103 and will not be described again.

[0106] In other embodiments, the specific implementation method of obtaining the image category using the image classification model can also refer to the embodiment in step S603 and will not be repeated.

[0107] The image classification method provided in the embodiments of this application can directly determine the image category of an input image based on an image classification model, such as whether it is an AIGI image. The image classification method provided in the embodiments of this application can be applied in a variety of application fields. For example, it can be used in the field of document anti-counterfeiting to determine whether a document uploaded by a user is a generated image. It can also be applied in the field of marketing content review, by detecting images in marketing materials to determine whether they are AI-generated, thereby ensuring that marketing content complies with relevant regulations and ethical standards and avoiding false advertising.

[0108] like Figure 9 The figure shows a functional module diagram of a model training device provided by an embodiment of the present application. The model training device 9 includes an acquisition unit 91, a fusion unit 92, a classification unit 93, a training unit 94, and an update unit 95. The module / unit referred to in this application refers to a type of unit that can be processed by a processor (e.g. Figure 10 The processor 1101 shown in FIG. 1 is obtained and is capable of performing a series of computer-readable instruction segments that are stored in a memory (eg, Figure 10 1102).

[0109] An acquisition unit 91 is configured to acquire a first eigenvector of the first image through an attention head in the first model, and to acquire a second eigenvector corresponding to the first eigenvector through a multilayer perceptron of the first model.

[0110] The fusion unit 92 is used to fuse the first feature vector and the second feature vector based on the weight of the attention head in the first model and the weight of the multi-layer perceptron of the first model to obtain a first fused feature.

[0111] The classification unit 93 is configured to classify the first image using the first model according to the first fusion feature to obtain a first predicted category.

[0112] The training unit 94 is configured to determine a first loss value of the first model according to the first prediction category, and train the first model based on the first loss value to obtain a second model.

[0113] An updating unit 95 is configured to determine a first score of the attention head in the second model based on the second image, and to update a weight of the attention head in the second model based on the first score.

[0114] Figure 10 Schematic diagram of the structure of a computer device for implementing the model training method provided in the embodiment of the present application. Figure 10 The computer device 10 shown is used to execute the methods in the above-mentioned method embodiments.

[0115] The computer device 10 includes at least one processor 1101 , a memory 1102 , and at least one network interface 1103 .

[0116] The processor 1101 is, for example, a general-purpose central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the solution of the present application. For example, the processor 1101 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0117] The memory 1102 may be, for example, a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Optionally, the memory 1102 exists independently and is connected to the processor 1101 via the internal connection 1104. Alternatively, the memory 1102 and the processor 1101 may be integrated together.

[0118] The network interface 1103 uses any transceiver-like device for communicating with other devices or communication networks. For example, the network interface 1103 includes at least one of a wired network interface and a wireless network interface. For example, the wired network interface is an Ethernet interface. For example, the Ethernet interface is an optical interface, an electrical interface, or a combination thereof. For example, the wireless network interface is a wireless local area network (WLAN) interface, a cellular network interface, or a combination thereof.

[0119] In some embodiments, the processor 1101 includes one or more CPUs, such as Figure 10 CPU0 and CPU1 are shown in the figure.

[0120] In some embodiments, the computer device 10 optionally includes multiple processors, such as Figure 10 1 and 1105. Each of these processors is, for example, a single-CPU or a multi-CPU. A processor herein optionally refers to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). In some embodiments, computer device 10 further includes internal connections 1104. Processor 1101, memory 1102, and at least one network interface 1103 are connected via internal connections 1104. Internal connections 1104 include pathways that transmit information between these components. Optionally, internal connections 1104 are boards or buses. Optionally, internal connections 1104 are divided into address buses, data buses, control buses, and the like.

[0121] In some embodiments, the computer device 10 further includes an input / output interface 1106 . The input / output interface 1106 is connected to the internal connection 1104 .

[0122] Optionally, the processor 1101 implements the method in the above embodiment by reading the program code 910 stored in the memory 1102, or the processor 1101 implements the method in the above embodiment by internally stored program code. In the case where the processor 1101 implements the method in the above embodiment by reading the program code 910 stored in the memory 1102, the memory 1102 stores the program code that implements the method provided in the embodiment of the present application.

[0123] For more details on how the processor 1101 implements the above functions, please refer to the descriptions in the previous method embodiments, which will not be repeated here.

[0124] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer device, the computer device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0125] This embodiment further provides a computer program product, including a computer program / instruction. When the computer program product is run on a computer device, the computer device is caused to execute the above-mentioned related steps to implement the methods in the above-mentioned method embodiments.

[0126] This embodiment also provides a chip system, which is applied to a computer device. The chip system includes one or more processors, which are used to call computer instructions to enable the computer device to input a first training data set into the chip system and execute the methods in the above-mentioned method embodiments.

[0127] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0128] Among them, the computer device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0129] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0132] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0134] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A model training method, characterized in that: The method comprises: Obtaining a first eigenvector of the first image through an attention head in the first model, and obtaining a second eigenvector corresponding to the first eigenvector through a multilayer perceptron of the first model; fusing the first feature vector and the second feature vector based on the weights of the attention heads in the first model and the weights of the multilayer perceptron of the first model to obtain a first fused feature; classifying the first image using the first model according to the first fusion feature to obtain a first predicted category; determining a first loss value of the first model according to the first prediction category, and training the first model based on the first loss value to obtain a second model; A first score of an attention head in the second model is determined based on the second image, and a weight of the attention head in the second model is updated based on the first score.

2. The method according to claim 1, characterized in that Obtaining a first feature vector of the first image by using an attention head in the first model includes: Dividing the first image into image blocks, and obtaining image block vectors corresponding to the image blocks; Performing linear mapping on the image block vector to obtain a first embedding vector; The first embedding vector is calculated using an attention head in the first model to obtain the first feature vector.

3. The method according to claim 1, characterized in that The obtaining of a second eigenvector corresponding to the first eigenvector by using the multilayer perceptron of the first model includes: The first eigenvector is subjected to a nonlinear transformation by a multilayer perceptron of the first model to obtain the second eigenvector.

4. The method according to claim 1, wherein The fusing the first feature vector and the second feature vector based on the weights of the attention heads in the first model and the weights of the multilayer perceptron of the first model includes: Using the weight of the attention head in the first model as the weight of the corresponding first feature vector; Using the weight of the multilayer perceptron of the first model as the weight of the corresponding second eigenvector; According to the weight of the first eigenvector and the weight of the second eigenvector, a weighted sum is performed on the first eigenvector and the second eigenvector, and the first fusion feature is obtained according to a result of the weighted sum.

5. The method according to claim 1, wherein Determining a first score of an attention head in the second model based on the second image includes: determining a second predicted category for the second image based on the attention heads in the second model; Determine a first score of an attention head of an encoder in the second model based on the precision and recall of the second predicted category.

6. The method according to claim 5, characterized in that Determining a second predicted category of the second image based on the attention head in the second model includes: Obtaining, through the attention head in the second model, a third eigenvector corresponding to the second image; Performing a nonlinear transformation on the third eigenvector using a multilayer perceptron of the second model to obtain a fourth eigenvector; classifying the second image using the second model according to the fourth eigenvector to obtain a second predicted category; The accuracy and recall of the second predicted category are determined according to the second predicted category and a preset category label of the second image.

7. The method according to claim 1, characterized in that The updating the weight of the attention head in the second model based on the first score includes: determining a first score for each attention head in the second model; Selecting an attention head from all attention heads in the second model according to the maximum value of the first scores; The weight of the selected attention head is updated so that the updated weight of the selected attention head is greater than the weight of the unselected attention heads.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining a fifth eigenvector of the third image by the attention head; A second score of the feature component of the fifth feature vector is determined, and a weight of the corresponding feature component is updated according to the second score.

9. The method according to claim 8, characterized in that Determining a second score of the feature component of the fifth feature vector includes: Performing a nonlinear transformation on the fifth eigenvector using the multilayer perceptron to obtain a corresponding sixth eigenvector; removing the characteristic component of the first dimension in the sixth eigenvector to obtain an updated sixth eigenvector; classifying the third image using the classifier of the model based on the updated sixth eigenvector to obtain a third predicted category; A score of the classifier is determined according to the third predicted category, and a second score of the feature component of the first dimension is determined based on the score of the classifier.

10. The method according to claim 9, characterized in that The updating of the weight of the corresponding feature component according to the second score includes: If the second score is less than a preset score threshold, the weight of the feature component of the first dimension is updated so that the updated weight of the feature component of the first dimension is less than the weights of the feature components of other dimensions.

11. The method according to claim 1, wherein The method further comprises: Obtaining a residual vector between the embedding vector of the fourth image and the embedding vector of the fifth image; predicting the category of the fifth image using the classifier according to the residual vector to obtain a third predicted category; A second loss value of the classifier is determined according to the third predicted category, and the classifier is trained based on the second loss value.

12. An image classification method, characterized in that: The method comprises: An image category corresponding to an input image is obtained using an image classification model, wherein the image classification model is obtained by executing the model training method according to any one of claims 1 to 11.

13. The method according to claim 12, characterized in that The image classification model is used to obtain the image category corresponding to the input data, including: Using the image classification model to segment the input image into image blocks, and obtaining image block vectors corresponding to the image blocks; Performing linear mapping on the image block vector to obtain an embedding vector; Calculating the embedding vector to obtain a first eigenvector; Performing a nonlinear transformation on the first eigenvector to obtain a second eigenvector, and fusing the first eigenvector and the second eigenvector to obtain a fused feature; The input image is classified based on the fusion features to obtain the image category.

14. A chip system, characterized in that: The chip system is applied to a computer device, and the chip system includes one or more processors, which are used to call computer instructions to enable the computer device to input a training data set into the chip system and execute the model training method as described in any one of claims 1 to 11, or the image classification method as described in any one of claims 12 to 13.

15. A computer device, characterized in that: The computer device includes a processor and a memory, and the processor is used to implement the model training method according to any one of claims 1 to 11, or the image classification method according to any one of claims 12 to 13 when executing the computer program stored in the memory.

Images