Knowledge distillation method, device and computer equipment for image classification model

Abstract

The present application relates to a knowledge distillation method, apparatus and computer equipment for an image classification model. The method obtains a training data set for knowledge distillation, inputs each sample image in the training data set into a first classification model for image feature extraction, and obtains an image feature set corresponding to the training data set. For any sample image in the training data set, image feature extraction is performed through a second classification model to obtain a corresponding second image feature, and a distillation loss is determined based on the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set. The second classification model is back-propagated based on the distillation loss, and the model parameters of the second classification model are updated, so that the second classification model can truly learn the relationship features between the sample images and achieve a prediction effect similar to that of the first classification model, so as to obtain an image classification model with better performance.

Description

Technical Field

[0001] The present application relates to the technical field of machine learning, and in particular to a knowledge distillation method, apparatus and computer equipment for an image classification model. Background Art

[0002] With the development of machine learning technology, it has become increasingly common to use models for image processing or recognition. Generally speaking, larger, deeper, and more complex models have better fitting effects and better prediction capabilities, but at the same time, they have low computational efficiency, high time consumption, and a large number of parameters, which is not conducive to deployment at the application layer such as mobile terminals and chip terminals; while simple models have weak fitting capabilities, but they have higher computational efficiency and fewer parameters, which is more conducive to deployment.

[0003] Knowledge distillation, as an important model compression method, can transfer the knowledge (dark knowledge) in a complex model (teacher, also known as the teacher module or the first model) to a simple model (student, also known as the student model or the second model) to make the student model's fitting ability approach or even exceed the teacher model, thereby obtaining similar prediction results with less time and space complexity. However, different knowledge distillation methods have different definitions of the dark knowledge that needs to be transferred. Among them, in classification tasks, traditional knowledge distillation based on output results usually directly shortens the distance between the output values ​​of the teacher model and the student model (the logits (logarithm of the ratio of an event occurring to the event not occurring) output after the model softmax (logistic regression) or the feature (feature) output by the middle layer of the network) so that the student model obtains prediction results that are similar to those of the teacher model.

[0004] However, current knowledge distillation methods based on image classification tasks usually only consider information extraction at the level of a single batch of samples, which is not global and thus cannot obtain an image classification student model with better performance. Summary of the invention

[0005] Based on this, it is necessary to provide a knowledge distillation method, device and computer equipment that can obtain an image classification model with better performance in order to address the problem that the traditional knowledge distillation based on output results cannot obtain a student model with better performance.

[0006] A knowledge distillation method for an image classification model, the method comprising:

[0007] Acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images;

[0008] Input each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one;

[0009] For any sample image in the training data set, extract image features through the second classification model to obtain corresponding second image features;

[0010] Determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set;

[0011] Back-propagation is performed on the second classification model according to the distillation loss to update model parameters of the second classification model.

[0012] In one of the embodiments, determining the distillation loss based on the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set includes: based on any sample image in the training data set, obtaining the positive feature and the negative feature in the image feature set corresponding to the sample image; and calculating the distillation loss based on the positive feature, the negative feature and the second image feature corresponding to the sample image.

[0013] In one of the embodiments, the method of obtaining positive features and negative features corresponding to the sample image in the image feature set based on any sample image in the training data set includes: obtaining a first image feature corresponding to the sample image in the image feature set based on any sample image in the training data set, and determining it as the positive feature corresponding to the sample image; obtaining other first image features in the image feature set except the positive feature corresponding to the sample image, and determining them as the negative feature corresponding to the sample image.

[0014] In one embodiment, the calculating the distillation loss according to the positive feature, the negative feature and the second image feature corresponding to the sample image includes: calculating the distillation loss using an information noise convergence estimation loss function according to the positive feature, the negative feature and the second image feature corresponding to the sample image, wherein the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i-is the negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

[0015] In one of the embodiments, the step of inputting each sample image in the training data set into the first classification model for image feature extraction to obtain an image feature set corresponding to the training data set includes: inputting each sample image in the training data set into the first classification model to obtain first original image features corresponding to each sample image; performing norm normalization processing on each first original image feature to obtain a corresponding processed first image feature; and obtaining a corresponding image feature set based on the first image features corresponding to each first original image feature.

[0016] In one of the embodiments, for any sample image in the training data set, image features are extracted through a second classification model to obtain corresponding second image features, including: for any sample image in the training data set, the sample image is input into a second classification model to obtain a second original image feature corresponding to the sample image; and the second original image feature is affine transformed and norm normalized to obtain a corresponding processed second image feature.

[0017] An image classification processing method, the method comprising:

[0018] Get the image to be processed;

[0019] The image to be processed is classified by an image classification model to obtain a classification result of the image to be processed, and the image classification model is a second classification model obtained by the knowledge distillation method of the image classification model as described above.

[0020] A knowledge distillation device for an image classification model, the device comprising:

[0021] A training data set acquisition module, used to acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images;

[0022] A first image feature extraction module, used for inputting each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one;

[0023] A second image feature extraction module, used for extracting image features of any sample image in the training data set through a second classification model to obtain corresponding second image features;

[0024] A distillation loss acquisition module, used to determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set;

[0025] A back propagation module is used to perform back propagation on the second classification model according to the distillation loss to update the model parameters of the second classification model.

[0026] An image classification processing device, the device comprising:

[0027] The module for acquiring the image to be processed is used to acquire the image to be processed;

[0028] A classification processing module is used to classify the image to be processed through an image classification model to obtain a classification result of the image to be processed. The image classification model is a second classification model obtained through the knowledge distillation method of the image classification model as described above.

[0029] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0030] A computer-readable storage medium stores a computer program, which implements the steps of the method described above when executed by a processor.

[0031] The knowledge distillation method, device and computer equipment of the above-mentioned image classification model obtain a training data set for knowledge distillation, input each sample image in the training data set into the first classification model for image feature extraction, and obtain an image feature set corresponding to the training data set. For any sample image in the training data set, image feature extraction is performed through the second classification model to obtain the corresponding second image feature, and the distillation loss is determined according to the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set. The second classification model is back-propagated according to the distillation loss, and the model parameters of the second classification model are updated, so that the second classification model can truly learn the relationship features between the sample images and achieve a prediction effect similar to that of the first classification model, so as to obtain an image classification model with better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of a process of a knowledge distillation method for an image classification model in one embodiment;

[0033] Figure 2 A schematic diagram of a process of obtaining a first image feature in an embodiment;

[0034] Figure 3A schematic diagram of a flow chart of a step of obtaining a second image feature in an embodiment;

[0035] Figure 4 A schematic diagram of a process for determining a distillation loss step in one embodiment;

[0036] Figure 5 A schematic diagram of a flow chart of a knowledge distillation method for an image classification model in another embodiment;

[0037] Figure 6 Schematic diagram of a flow chart of an image classification processing method in one embodiment;

[0038] Figure 7 is a structural block diagram of a knowledge distillation device for an image classification model in one embodiment;

[0039] Figure 8 is a structural block diagram of an image classification processing device in one embodiment;

[0040] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] Since traditional knowledge distillation based on output results directly shortens the distance between the output values ​​of the teacher model and the student model, the student model can obtain prediction results that are close to the teacher model. However, this method obviously has intuitive shortcomings. The student model can only learn the output performance of the teacher model, but cannot truly learn the relationship characteristics between samples. Recently, some knowledge in the field of metric learning has also been slowly used to improve the loss function in knowledge distillation. The intuitive understanding is that samples that are close in the feature space of the teacher model should also be close enough in the feature space of the student model. Similarly, samples that are far in the feature space of the teacher model should also be far away in the feature space of the student model.

[0043] Based on this, this application provides a knowledge distillation method for image classification models, such as Figure 1 As shown, the following steps are included:

[0044] Step 110, obtaining a training data set for knowledge distillation.

[0045] Among them, the training data set includes multiple sample images, and the training data set is used to transfer the knowledge in the teacher model to the student model through knowledge distillation, so that the student model can truly learn the relationship features between the sample images, thereby achieving a similar prediction effect as the teacher model. Sample data. In this embodiment, taking the image classification task as an example, the teacher model can be a first classification model whose model parameters have been trained, and the student model can be a second classification model whose knowledge in the first classification model is to be obtained through knowledge distillation. Specifically, when the knowledge in the first classification model is to be transferred to the second classification model through knowledge distillation, it is first necessary to obtain a training data set for knowledge distillation. Each sample image in the training data set can be a static image or video frame image of any size.

[0046] Step 120: Input each sample image in the training data set into the first classification model to extract image features, and obtain an image feature set corresponding to the training data set.

[0047] Among them, the first classification model is a trained neural network for classifying images, which usually includes an input layer, a hidden layer and an output layer. In this embodiment, the first classification model is a neural network with a certain complexity and scale. The image feature set includes a first image feature corresponding to each sample image one by one. The first image feature is the image feature result extracted by the first classification model when classifying and detecting the sample image. In this embodiment, the first image feature is the image feature obtained before the sample image passes through the output layer of the first classification model. Specifically, each sample image in the training data set is input into the first classification model to obtain the first image feature corresponding to each sample image. Based on the first image feature corresponding to each sample image in the training data set, an image feature set corresponding to the training data set is obtained, that is, the image feature set includes the first image feature of each sample image.

[0048] Step 130: For any sample image in the training data set, image feature extraction is performed using a second classification model to obtain corresponding second image features.

[0049] Among them, the second classification model is a neural network that is to obtain the knowledge in the above-mentioned first classification model through knowledge distillation, which generally includes an input layer, a hidden layer and an output layer. The complexity and scale of the second classification model are lower than those of the above-mentioned first classification model. That is to say, the first classification model can be a neural network with more layers, complex structure and powerful functions, while the second classification model is a neural network with fewer layers and simple structure. The second image feature is the image feature result extracted by the second classification model when classifying and detecting the sample image. In this embodiment, the second image feature is the image feature obtained before the sample image passes through the output layer of the second classification model. Specifically, for any sample image in the training data set, the sample image is input into the second classification model to obtain the second image feature corresponding to the sample image.

[0050] Step 140 , determining the distillation loss according to the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set.

[0051] The distillation loss is used to characterize the gap between the second classification model and the first classification model during the knowledge distillation process. In this embodiment, the distillation loss is determined by the second image features corresponding to any sample image in the training data set and the image feature set corresponding to the training data set, and then the model parameters of the second classification model are adjusted based on the distillation loss, so that the second classification model can truly learn the relationship features between the sample images and achieve a prediction effect similar to that of the first classification model.

[0052] Step 150, back-propagating the second classification model according to the distillation loss to update the model parameters of the second classification model.

[0053] Specifically, after the distillation loss is obtained through the above steps, the second classification model can be back-propagated once according to the distillation loss to complete the calculation of an iterative training of the second classification model, thereby updating the model parameters of the second classification model. It can be understood that for each sample image in the training data set, the second image features can be extracted according to the method of steps 130 to 150 above, and the distillation loss can be determined, and then the model parameters of the second classification model can be updated according to the distillation loss, so that the second classification model can truly learn the relationship features between the sample images and achieve a prediction effect similar to that of the first classification model.

[0054] The knowledge distillation method of the above-mentioned image classification model obtains a training data set for knowledge distillation, inputs each sample image in the training data set into the first classification model for image feature extraction, and obtains an image feature set corresponding to the training data set. For any sample image in the training data set, image feature extraction is performed through the second classification model to obtain the corresponding second image feature, and the distillation loss is determined according to the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set. The second classification model is back-propagated according to the distillation loss, and the model parameters of the second classification model are updated, so that the second classification model can truly learn the relationship features between the sample images and achieve a prediction effect similar to that of the first classification model, so as to obtain an image classification model with better performance.

[0055] In one embodiment, Figure 2 As shown, in the above step 120, each sample image in the training data set is input into the first classification model to extract image features to obtain an image feature set corresponding to the training data set, which may specifically include the following steps:

[0056] Step 121: input each sample image in the training data set into the first classification model to obtain a first original image feature corresponding to each sample image.

[0057] The first original image feature is the original image feature obtained before the sample image passes through the output layer of the first classification model, that is, after the sample image passes through the hidden layer of the first classification model but has not been processed by the output layer. Specifically, by inputting each sample image in the training data set into the first classification model, the first original image feature corresponding to each sample image is obtained, that is, the first original image feature corresponding to each sample image is obtained one by one.

[0058] Step 122, performing norm normalization processing on each first original image feature to obtain a corresponding processed first image feature.

[0059] Among them, the norm normalization processing in this embodiment can be processed by L2 norm normalization. L2 norm normalization is to divide each dimensional data x1, x2, ..., xn of vector X by ||x||2 (i.e., L2 norm) to obtain a new vector. The L2 norm normalization processing can eliminate the influence of units between features and improve the convergence speed of the learning model. In this embodiment, each dimensional data of each first original image feature is divided by the L2 norm, so as to obtain the first image feature after the norm normalization processing of each first original image feature.

[0060] Step 123: Obtain a corresponding image feature set based on the first image features corresponding to each first original image feature.

[0061] Specifically, based on the first image features corresponding to each first original image feature, an image feature set corresponding to the training data set is obtained, that is, the image feature set includes the first image features corresponding to each sample image one by one.

[0062] In the above embodiment, by inputting each sample image in the training data set into the first classification model, a first original image feature corresponding to each sample image is obtained, and norm normalization processing is performed on each first original image feature to obtain the corresponding processed first image feature. Based on the first image features corresponding to each first original image feature, a corresponding image feature set is obtained. Since the image feature set stores image features of multiple sample images, the problem of insufficient number of samples in a single batch is solved.

[0063] In one embodiment, Figure 3 As shown, in the above step 130, for any sample image in the training data set, image feature extraction is performed through the second classification model to obtain the corresponding second image feature, which may specifically include the following steps:

[0064] Step 131: for any sample image in the training data set, the sample image is input into a second classification model to obtain a second original image feature corresponding to the sample image.

[0065] Similarly, the second original image feature is the original image feature obtained before the sample image passes through the output layer of the second classification model, that is, after the sample image passes through the hidden layer of the second classification model but has not yet been processed by the output layer. Specifically, for any sample image in the training data set, the second original image feature corresponding to the sample image is obtained by inputting the sample image into the second classification model.

[0066] Step 132, performing affine transformation and norm normalization processing on the second original image feature to obtain a corresponding processed second image feature.

[0067] Among them, affine transformation, also known as affine mapping, refers to the process of transforming a vector space into another vector space by performing a linear transformation and a translation in geometry. In this embodiment, by performing an affine transformation on the second original image feature, the dimension of the second original image feature output by the second classification model is made consistent with the dimension of the first original image feature output by the first classification model. Then, the second original image feature after affine transformation is subjected to norm normalization processing to obtain the corresponding processed second image feature. Among them, the norm normalization processing can adopt the same method as in the above step 122, which will not be repeated in this embodiment.

[0068] In the above embodiment, for any sample image in the training data set, the sample image is input into the second classification model to obtain a second original image feature corresponding to the sample image, and the second original image feature is subjected to affine transformation and norm normalization processing to obtain a second image feature having the same dimension as the first image feature after corresponding processing, thereby facilitating the second classification model to learn the relationship features between the sample images.

[0069] In one embodiment, Figure 4 As shown, in the above step 140, the distillation loss is determined according to the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set, which may specifically include the following steps:

[0070] Step 141 , based on any sample image in the training data set, obtain positive feature and negative feature corresponding to the sample image in the image feature set.

[0071] Among them, the positive feature refers to the first image feature corresponding to any sample image in the training data set in the image feature set, and the negative feature refers to the other first image features in the image feature set except the positive feature corresponding to the sample image. That is, for any sample image in the training data set, the corresponding positive feature is the first image feature corresponding to the sample image in the image feature set, and the negative feature is all the first image features in the image feature set except the positive feature corresponding to the sample image. Based on this, the positive feature and negative feature corresponding to any sample image in the training data set can be determined.

[0072] Step 142 calculates the distillation loss according to the positive feature, the negative feature and the second image feature corresponding to the sample image.

[0073] Specifically, in this embodiment, the information noise convergence estimation loss function (info Noise-Constrastive Estimation Loss) can be used to calculate the distillation loss, wherein the information noise convergence estimation loss function Among them, q is the second image feature corresponding to any sample image in the training data set, t + is the positive feature corresponding to the sample image in the image feature set, t i- is the i-th negative example feature corresponding to the sample image in the image feature set, K is the number of negative example features corresponding to the sample image in the image feature set, and τ is a configurable hyperparameter.

[0074] like Figure 5 As shown, the following is based on Figure 5 The knowledge distillation method of the image classification model in the present application is further described, wherein the model structures of the Teacher model and the Student model are predetermined, and the network parameters of the Teacher model are pre-trained, and the method for performing knowledge distillation on the Student model is as follows:

[0075] 1. Fix the network parameters of the Teacher model, obtain the training data set for knowledge distillation (assuming it contains N sample images), input each sample image into the Teacher model, and obtain the feature of each sample image before passing through the fully connected layer of the Teacher model (i.e., the first original image feature), which are recorded as f1, f2, …, f N , where f i ∈R D , D is the feature dimension. For each sample image output feature f i (i.e., the first original image feature corresponding to each sample image) is subjected to L2 norm normalization processing via the "L2 norm 1" layer (i.e., the L2 norm normalization layer), thereby obtaining the corresponding processed first image feature, i.e., t i .

[0076] Among them, the processing process of the L2 norm normalization layer (i.e. L2 norm 1) is as follows:

[0077] Among them, f i is the first original image feature obtained after the output of the Teacher model for the i-th sample image, ‖f i ‖2 is the feature f output by the i-th sample image i The corresponding L2 norm, D is the feature dimension, f ij f i The j-th dimension image feature of .

[0078] Among them, t i Then for f i The first image feature is obtained after L2 norm normalization. For each sample image in the above training data set, f1, f2, ..., f N , after L2 norm normalization, we get the corresponding t1, t2, …, t N , the image feature set composed of is used as a Memory Bank, whose size is N×D, where N is the number of elements in the set, that is, the number of first image features, and D is the feature dimension of each first image feature.

[0079] 2. For any sample image m in the training data set, obtain the feature of the sample image m before passing through the fully connected layer of the Student model (i.e., the second original image feature), denoted as f m , then f m ∈R d , d is the feature dimension.

[0080] In this embodiment, by m Perform an affine transformation (specifically, the affine transformation can be performed through the FC layer in the figure), so that f m With f i For the same feature dimension, in this embodiment, the specific affine transformation of the FC layer is as follows:

[0081] q m ′=Wf m +b; where W∈R d×D , W represents the parameter matrix of affine transformation, b is the bias, and d is f m The feature dimension of f m The target feature dimension for affine transformation, f m is the second original image feature obtained after the sample image m is output by the Student model, q m ′ is the pair f m The corresponding image features obtained after affine transformation.

[0082] Then, q m ' performs L2 norm normalization processing (specifically processed by the "L2 norm 2" layer), thereby obtaining the corresponding processed second image feature, i.e., q. The L2 norm normalization processing process is as follows:

[0083] Among them, q m ′ is the pair f m The corresponding image features obtained after affine transformation, ‖q m ′‖2 is the image feature qm ′ corresponds to the L2 norm, D is the feature dimension, q m ' j Q m ′ is the j-th dimension image feature.

[0084] Among them, q is the m ′ is the second image feature obtained after L2 norm normalization.

[0085] 3. For sample image m, it corresponds to the first image feature t in Memory Bank i+ is the positive example feature (for the convenience of explanation, t is used in the following formula + ), while other first image features in Memory Bank is the negative feature corresponding to the sample image m (for the convenience of explanation, t is used in the following formula i- , where K is the number of negative features.

[0086] 4. For the sample image m, the infoNCE loss corresponding to the sample image is calculated based on the positive and negative features in the corresponding Memory Bank obtained above:

[0087] Among them, q is the second image feature corresponding to the sample image m, t + is the positive feature corresponding to the sample image m, t i- is the i-th negative feature corresponding to the sample image m, K is the number of negative features, and τ is a configurable hyperparameter.

[0088] Based on this, for any sample image n, the infoNCE loss is calculated based on the positive and negative features in the Memory Bank:

[0089] Among them, q n is the second image feature corresponding to any sample image n, t n+ is the positive feature corresponding to the sample image n, is the jth negative feature corresponding to sample image n, K is the number of negative features, and τ is a configurable hyperparameter.

[0090] 5. Loss is returned and the student network parameters are updated according to the gradient.

[0091] The knowledge distillation method for the image classification model provided in this embodiment introduces a memory bank to store feature vectors of a large number of sample images, thereby solving the problem of insufficient number of sample images in a single batch. At the same time, the KL divergence commonly used in traditional knowledge distillation is replaced by infoNCE loss. Compared with the traditional relationship distillation solution, the infoNCE loss with excellent performance in representation learning tasks is used to extract the similarity information between the sample images output by the Teacher model and the Student model, and the Memory Bank is used to ensure feature reuse. At the same time, the excellent ability of infoNCE to distinguish Positive Pair (positive example features) from Negative Pair (negative example features) is used, and the large number of Negative Pairs provided by the Memory Bank, thereby greatly improving the knowledge distillation effect of the image classification model.

[0092] In one embodiment, Figure 6 As shown, the present application also provides an image classification processing method, comprising the following steps:

[0093] Step 610: Obtain an image to be processed.

[0094] The image to be processed is an image to be classified, which includes but is not limited to scenes involving classification in target detection or recognition, foreground and background segmentation, or target tracking.

[0095] Step 620: classify the image to be processed using the image classification model to obtain a classification result of the image to be processed.

[0096] Among them, the image classification model is as above Figures 1 to 5 The second classification model is obtained by the knowledge distillation method of the image classification model. It can be understood that Figures 1 to 5 After the knowledge distillation method of the image classification model obtains the second classification model, the second classification model can also be trained based on the scenarios involving classification in actual applications, so that it can learn the classification features in the specific scenarios, and classify the image to be processed through the trained second classification model to obtain the classification result of the image to be processed.

[0097] Since the image classification model used for classification processing is Figures 1 to 5 The image classification model is obtained by the knowledge distillation method. Therefore, the image classification model can truly learn the relationship features between sample images and achieve a prediction effect similar to that of a complex model. After being trained and used for specific classification tasks, the classification results it outputs will also be more accurate.

[0098] It should be understood that although Figure 1-6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0099] In one embodiment, Figure 7 As shown, a knowledge distillation device for an image classification model is provided, comprising: a training data set acquisition module 701, a first image feature extraction module 702, a second image feature extraction module 703, a distillation loss acquisition module 704 and a back propagation module 705, wherein:

[0100] A training data set acquisition module 701 is used to acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images;

[0101] A first image feature extraction module 702 is used to input each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one;

[0102] A second image feature extraction module 703 is used to extract image features from any sample image in the training data set through a second classification model to obtain corresponding second image features;

[0103] A distillation loss acquisition module 704, configured to determine a distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set;

[0104] The back propagation module 705 is used to perform back propagation on the second classification model according to the distillation loss to update the model parameters of the second classification model.

[0105] In one embodiment, the distillation loss acquisition module 704 includes: a feature acquisition unit, used to obtain positive features and negative features corresponding to the sample image in the image feature set based on any sample image in the training data set; a loss calculation unit, used to calculate the distillation loss according to the positive features, negative features and second image features corresponding to the sample image.

[0106] In one embodiment, the feature acquisition unit is specifically used to: based on any sample image in the training data set, obtain the first image feature corresponding to the sample image in the image feature set, and determine it as the positive example feature corresponding to the sample image; obtain other first image features in the image feature set except the positive example feature corresponding to the sample image, and determine them as the negative example features corresponding to the sample image.

[0107] In one embodiment, the loss calculation unit is specifically used to: calculate the distillation loss using an information noise convergence estimation loss function according to the positive example features, negative example features and second image features corresponding to the sample image, wherein the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i- is the i-th negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

[0108] In one embodiment, the first image feature extraction module is specifically used to: input each sample image in the training data set into the first classification model to obtain a first original image feature corresponding to each sample image; perform norm normalization processing on each first original image feature to obtain the corresponding processed first image feature; based on the first image features corresponding to each first original image feature, obtain a corresponding image feature set.

[0109] In one embodiment, the second image feature extraction module is specifically used to: for any sample image in the training data set, input the sample image into the second classification model to obtain a second original image feature corresponding to the sample image; perform affine transformation and norm normalization processing on the second original image feature to obtain the corresponding processed second image feature.

[0110] For the specific definition of the knowledge distillation device of the image classification model, please refer to the definition of the knowledge distillation method of the image classification model mentioned above, which will not be repeated here. Each module in the knowledge distillation device of the above-mentioned image classification model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0111] In one embodiment, Figure 8 As shown, an image classification processing device is provided, comprising: a to-be-processed image acquisition module 801 and a classification processing module 802, wherein:

[0112] The image to be processed acquisition module 801 is used to acquire the image to be processed;

[0113] The classification processing module 802 is used to classify the image to be processed through an image classification model to obtain a classification result of the image to be processed. The image classification model is a second classification model obtained through the knowledge distillation method of the image classification model as described above.

[0114] For the specific definition of the image classification processing device, please refer to the definition of the image classification processing method above, which will not be repeated here. Each module in the above-mentioned image classification processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0115] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a set of image features corresponding to a training data set. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a knowledge distillation method of an image classification model or an image classification processing method is implemented.

[0116] Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0118] Acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images;

[0119] Input each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one;

[0120] For any sample image in the training data set, extract image features through the second classification model to obtain corresponding second image features;

[0121] Determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set;

[0122] Back-propagation is performed on the second classification model according to the distillation loss to update model parameters of the second classification model.

[0123] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on any sample image in the training data set, obtaining positive features and negative features corresponding to the sample image in the image feature set; and calculating the distillation loss according to the positive features, negative features and second image features corresponding to the sample image.

[0124] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on any sample image in the training data set, a first image feature in the image feature set corresponding to the sample image is obtained, and determined as a positive example feature corresponding to the sample image; other first image features in the image feature set except the positive example feature corresponding to the sample image are obtained, and determined as negative example features corresponding to the sample image.

[0125] In one embodiment, when the processor executes the computer program, the following steps are further implemented: according to the positive example features, the negative example features and the second image features corresponding to the sample image, the distillation loss is calculated using an information noise convergence estimation loss function, wherein the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i- is the i-th negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

[0126] In one embodiment, when the processor executes the computer program, the following steps are also implemented: each sample image in the training data set is input into the first classification model to obtain a first original image feature corresponding to each sample image; each first original image feature is subjected to norm normalization processing to obtain a corresponding processed first image feature; based on the first image features corresponding to each first original image feature, a corresponding image feature set is obtained.

[0127] In one embodiment, when the processor executes the computer program, the following steps are also implemented: for any sample image in the training data set, the sample image is input into a second classification model to obtain a second original image feature corresponding to the sample image; and the second original image feature is affine transformed and norm normalized to obtain a corresponding processed second image feature.

[0128] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0129] Get the image to be processed;

[0130] The image to be processed is classified by an image classification model to obtain a classification result of the image to be processed, and the image classification model is a second classification model obtained by the knowledge distillation method of the image classification model as described above.

[0131] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0132] Acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images;

[0133] Input each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one;

[0134] For any sample image in the training data set, extract image features through the second classification model to obtain corresponding second image features;

[0135] Determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set;

[0136] Back-propagation is performed on the second classification model according to the distillation loss to update model parameters of the second classification model.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on any sample image in the training data set, obtaining positive features and negative features corresponding to the sample image in the image feature set; and calculating the distillation loss according to the positive features, negative features and second image features corresponding to the sample image.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on any sample image in the training data set, a first image feature in the image feature set corresponding to the sample image is obtained, and determined as a positive example feature corresponding to the sample image; other first image features in the image feature set except the positive example feature corresponding to the sample image are obtained, and determined as negative example features corresponding to the sample image.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: according to the positive example features, the negative example features and the second image features corresponding to the sample image, the distillation loss is calculated using an information noise convergence estimation loss function, wherein the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i- is the i-th negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: each sample image in the training data set is input into the first classification model to obtain a first original image feature corresponding to each sample image; each first original image feature is subjected to norm normalization processing to obtain a corresponding processed first image feature; based on the first image feature corresponding to each first original image feature, a corresponding image feature set is obtained.

[0141] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: for any sample image in the training data set, the sample image is input into a second classification model to obtain a second original image feature corresponding to the sample image; and the second original image feature is affine transformed and norm normalized to obtain a corresponding processed second image feature.

[0142] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0143] Get the image to be processed;

[0144] The image to be processed is classified by an image classification model to obtain a classification result of the image to be processed, and the image classification model is a second classification model obtained by the knowledge distillation method of the image classification model as described above.

[0145] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A knowledge distillation method for an image classification model, characterized in that: The method comprises: Acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images; Input each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one; For any sample image in the training data set, extract image features through the second classification model to obtain corresponding second image features; Determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set; Performing backpropagation on the second classification model according to the distillation loss to update model parameters of the second classification model; The determining the distillation loss according to the second image feature corresponding to any sample image in the training data set and the image feature set corresponding to the training data set includes: Based on any sample image in the training data set, obtaining positive features and negative features corresponding to the sample image in the image feature set; According to the positive example features, negative example features and second image features corresponding to the sample image, the distillation loss is calculated using an information noise convergence estimation loss function, wherein the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i- is the i-th negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

2. The method according to claim 1, characterized in that The acquiring, based on any one sample image in the training data set, positive feature and negative feature corresponding to the sample image in the image feature set comprises: Based on any sample image in the training data set, obtaining a first image feature in the image feature set corresponding to the sample image, and determining it as a positive example feature corresponding to the sample image; Other first image features in the image feature set except the positive example features corresponding to the sample image are obtained and determined as negative example features corresponding to the sample image.

3. The method according to any one of claims 1 or 2, characterized in that: The step of inputting each sample image in the training data set into a first classification model to extract image features to obtain an image feature set corresponding to the training data set includes: Input each sample image in the training data set into a first classification model to obtain a first original image feature corresponding to each sample image; Performing norm normalization processing on each first original image feature to obtain a corresponding processed first image feature; Based on the first image features respectively corresponding to each first original image feature, a corresponding image feature set is obtained.

4. The method according to any one of claims 1 or 2, characterized in that: For any sample image in the training data set, extracting image features through a second classification model to obtain corresponding second image features includes: For any sample image in the training data set, input the sample image into a second classification model to obtain a second original image feature corresponding to the sample image; Affine transformation and norm normalization processing are performed on the second original image feature to obtain a corresponding processed second image feature.

5. An image classification processing method, characterized in that: The method comprises: Get the image to be processed; The image to be processed is classified by an image classification model to obtain a classification result of the image to be processed, wherein the image classification model is a second classification model obtained by the knowledge distillation method of the image classification model as described in any one of claims 1 to 4.

6. A knowledge distillation device for an image classification model, characterized in that: The device comprises: A training data set acquisition module, used to acquire a training data set for knowledge distillation, wherein the training data set includes a plurality of sample images; A first image feature extraction module, used for inputting each sample image in the training data set into a first classification model to extract image features, so as to obtain an image feature set corresponding to the training data set, wherein the image feature set includes a first image feature corresponding to each sample image one by one; A second image feature extraction module, used for extracting image features of any sample image in the training data set through a second classification model to obtain corresponding second image features; A distillation loss acquisition module, used to determine the distillation loss according to a second image feature corresponding to any sample image in the training data set and an image feature set corresponding to the training data set; A back propagation module, used to perform back propagation on the second classification model according to the distillation loss, and update the model parameters of the second classification model; The distillation loss acquisition module is specifically used to: based on any sample image in the training data set, obtain the positive feature and the negative feature corresponding to the sample image in the image feature set; according to the positive feature, the negative feature and the second image feature corresponding to the sample image, calculate the distillation loss using the information noise convergence estimation loss function, and the information noise convergence estimation loss function Wherein, q is the second image feature corresponding to the sample image, t + is the positive feature corresponding to the sample image, t i- is the i-th negative example feature corresponding to the sample image, K is the number of the negative example features, and τ is a hyperparameter.

7. An image classification processing device, characterized in that: The device comprises: The module for acquiring the image to be processed is used to acquire the image to be processed; A classification processing module is used to classify the image to be processed through an image classification model to obtain a classification result of the image to be processed, wherein the image classification model is a second classification model obtained by the knowledge distillation method of the image classification model as described in any one of claims 1 to 4.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images