Image classification method and system based on two-stage adaptive knowledge distillation

Abstract

The invention belongs to the technical field of image processing, and provides an image classification method and system based on dual-stage adaptive knowledge distillation, and the method comprises the steps: obtaining a to-be-classified image; based on a student model and a teacher model, extracting image local features and image global features of the obtained to-be-classified image; and processing the image to be classified according to the acquired image local features, the acquired image global features and the dual-stage adaptive knowledge distillation to complete image classification. According to the method, image classification is carried out by adopting two-stage self-adaptive knowledge distillation, through noise perception self-adaptive knowledge distillation, noise perception is introduced, the negative influence of teacher noise on student ability improvement is reduced, weight distribution of differentiated knowledge and redundant knowledge is effectively balanced, and the learning effect of a student model is improved; multivariate knowledge of different perspectives is integrated based on multivariate knowledge self-adaptive collaborative distillation, multi-dimensional optimization of a student model is realized, and the learning ability and generalization ability of the model are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image classification method and system based on two-stage adaptive knowledge distillation. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Image classification is one of the important research directions in the field of computer vision. Its purpose is to train network models so that they can automatically identify and accurately classify input images into predefined categories. With the development of deep learning, especially the emergence of convolutional neural networks, the development of computer vision has made great breakthroughs, and image classification technology has gradually matured and has been widely used in medical imaging diagnosis, autonomous driving, security monitoring and other fields. Thanks to its ability to automatically learn hierarchical features in images, deep learning methods have significantly improved the classification accuracy of the model and have become the mainstream method in the field of image classification.

[0004] However, existing deep learning methods can often achieve excellent performance, but the model size also increases, which greatly increases the model's inference time and computing resource utilization, which is not conducive to the deployment of deep learning models in real life.

[0005] As a model compression technology, knowledge distillation provides a new idea for model lightweighting. In this field, feature-based knowledge distillation is a commonly used knowledge distillation method. However, this type of method often does not divide knowledge finely enough during distillation and ignores the processing of noise. In fact, from the perspective of knowledge type, knowledge can be divided into global feature knowledge and local feature knowledge. According to whether there is noise in the knowledge generated by the teacher model, it can be further divided into discriminative knowledge and redundant knowledge. Discriminative knowledge often contributes more to model decision-making, while redundant features refer to knowledge that has negative feedback on the results, and this feature often contains a lot of noise.

[0006] If discriminative knowledge and redundant knowledge are treated uniformly and learned in the same way, some erroneous feature knowledge will be passed on to students. The learning of this feature knowledge is not conducive to the improvement of students' models. Instead, it will mislead students and have a negative impact on the final results.

[0007] In addition, existing knowledge distillation methods usually adopt a unified feature transfer strategy, assuming that all samples have the same absorption capacity and demand for the teacher model knowledge. However, this approach that ignores the personalized needs of samples may cause the student model to be unable to fully learn knowledge that is beneficial to its specific samples, thereby limiting the overall effect of distillation. Summary of the invention

[0008] To solve the above problems, the present invention proposes an image classification method and system based on two-stage adaptive knowledge distillation, which adopts two-stage adaptive knowledge distillation for image classification, introduces noise perception to determine the noise position through noise-aware adaptive knowledge distillation, and reduces the negative impact of teacher noise on student ability improvement. Through the progressive weight adjustment strategy, the weight distribution of discriminative knowledge and redundant knowledge is effectively balanced, the distillation process is optimized, and the learning effect of the student model is improved; based on the second stage of multi-knowledge adaptive collaborative distillation, multi-knowledge from different perspectives is integrated to achieve multi-dimensional optimization of the student model, improve the sum capacity of the student model, reduce redundant interference, and effectively improve the learning ability and generalization ability of the model.

[0009] According to some embodiments, a first solution of the present invention provides an image classification method based on two-stage adaptive knowledge distillation, which adopts the following technical solution: An image classification method based on two-stage adaptive knowledge distillation, comprising: Get the image to be classified; Based on the student model and the teacher model, extract the local image features and the global image features of the acquired image to be classified; According to the acquired local features and global features of the image, as well as the two-stage adaptive knowledge distillation, the image to be classified is processed to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

[0010] As a further technical limitation, the extracted local features and global features of the image are subjected to noise perception and knowledge decoupling respectively to obtain discriminative local features, redundant local features, discriminative global features and redundant global features; the obtained discriminative local features, redundant local features, discriminative global features and redundant global features are subjected to adaptive knowledge distillation to complete the noise-aware adaptive knowledge distillation of local features and global features of the image.

[0011] Furthermore, in the process of the adaptive knowledge distillation, mean square error loss function and mean absolute error are introduced for the discriminative knowledge and redundant features respectively, and the weight distribution of the discriminative knowledge and redundant knowledge is balanced through progressive adaptive weight adjustment, the adaptive distillation process is optimized, and the discriminative image local feature knowledge distillation loss function, the redundant image local feature knowledge distillation loss function, the discriminative image global feature knowledge distillation loss function and the redundant image global feature knowledge distillation loss function are calculated; logical knowledge distillation is performed on the extracted local image features and image global features respectively, and the logical knowledge distillation loss function is calculated.

[0012] Furthermore, logical knowledge distillation is performed on the extracted local features and global features of the image respectively, and cross entropy is used as the logical knowledge distillation loss function. Backpropagation is continuously performed. When the logical knowledge distillation loss function is minimized, the probability distributions of the teacher model and the student model are aligned to obtain the logical knowledge distillation loss function.

[0013] Furthermore, based on the results of the adaptive knowledge distillation and the logical knowledge distillation, multi-knowledge adaptive collaborative distillation is performed; a loss-contribution evaluation mechanism is adopted to dynamically evaluate the contribution of each type of knowledge and adjust its loss function weight by back-propagating the gradients of each distillation loss function.

[0014] Furthermore, based on the obtained discriminative image local feature knowledge distillation loss function, redundant image local feature knowledge distillation loss function, discriminative image global feature knowledge distillation loss function, redundant image global feature knowledge distillation loss function and logical knowledge distillation loss function, the overall distillation loss function and the gradient of each distillation loss function relative to the overall distillation loss function are calculated to obtain the weight of each distillation loss function, and the weighted sum of each distillation loss function is performed to obtain the total distillation loss function, thus completing the two-stage adaptive knowledge distillation.

[0015] According to some embodiments, a second solution of the present invention provides an image classification system based on two-stage adaptive knowledge distillation, which adopts the following technical solution: An image classification system based on two-stage adaptive knowledge distillation, comprising: An acquisition module, configured to acquire an image to be classified; An extraction module, which is configured to extract local image features and global image features of the acquired image to be classified based on the student model and the teacher model; A classification module is configured to process the image to be classified according to the acquired local features and global features of the image and the two-stage adaptive knowledge distillation to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

[0016] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of an image classification method based on two-stage adaptive knowledge distillation as described in the first solution of the present invention.

[0017] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution: An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in an image classification method based on two-stage adaptive knowledge distillation as described in the first scheme of the present invention are implemented.

[0018] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution: A computer program product includes software codes, wherein the program in the software codes executes the steps in an image classification method based on two-stage adaptive knowledge distillation as described in the first solution of the present invention.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention applies a variety of knowledge in multiple distillation processes, which not only enriches the learning content of the student model, but also greatly explores the utilization potential of feature knowledge; at the same time, it introduces a noise sensor to determine the noise position, ensure the quality of the distilled feature knowledge, and reduce the negative impact of teacher noise on student ability improvement. Through the progressive weight adjustment strategy, it can effectively balance the weight distribution of discriminative knowledge and redundant knowledge, thereby optimizing the distillation process and improving the learning effect of the student model.

[0020] The adaptive collaborative distillation of multivariate knowledge in the present invention satisfies the differentiated needs of samples. It can better integrate multivariate knowledge from different perspectives and ensure that the student model can be optimized in multiple dimensions. This synergy can effectively improve the comprehensive ability of the student model, especially when processing complex and diverse data; adaptive weighting can more flexibly respond to the diverse needs of various samples, reduce redundant interference, and effectively improve the learning ability and generalization performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0022] Figure 1 This is a flowchart of an image classification method based on two-stage adaptive knowledge distillation in Embodiment 1 of the present invention; Figure 2 Schematic diagram of a network learning process based on two-stage adaptive knowledge distillation in Embodiment 1 of the present invention; Figure 3 This is a structural block diagram of an image classification system based on two-stage adaptive knowledge distillation in Example 2 of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0025] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0026] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0027] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific research or technical personnel in this field, the specific meanings of the above terms in the present invention can be determined according to specific circumstances, and they cannot be understood as limitations on the present invention.

[0028] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0029] Embodiment 1 Embodiment 1 of the present invention introduces an image classification method based on two-stage adaptive knowledge distillation.

[0030] like Figure 1 An image classification method based on two-stage adaptive knowledge distillation is shown, comprising: Get the image to be classified; Based on the student model and the teacher model, extract the local image features and the global image features of the acquired image to be classified; According to the acquired local features and global features of the image, as well as the two-stage adaptive knowledge distillation, the image to be classified is processed to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

[0031] This embodiment combines Figure 2 The two-stage adaptive knowledge distillation network learning process shown in the figure is described in detail: Teacher model pre-training The teacher model is a hybrid architecture that includes local feature extractors and global feature extractors. Through convolution and attention mechanisms, the model can extract fine-grained feature knowledge through convolution and global features through attention mechanisms. To improve versatility, the teacher model backbone network is trained using various public data sets. After the model converges, its weight file is saved. For different downstream tasks, its original classification head weights are frozen, and the classification head is retrained. It is subsequently used to guide the training of the student model in specific downstream tasks.

[0032] Building a student model The student model also contains both local feature extractors and global feature extractors. Compared with the teacher model, the student model has fewer convolutional layers and attention blocks with smaller parameters, making it a lightweight model.

[0033] Dataset preprocessing During the data collection process, the image sizes of the original data samples may be inconsistent. In order to make the deep network model learn more effectively, we first perform data consistency operations on the original data sample set and adjust the image sizes uniformly. Specifically, we use the transforms class in the open source deep learning framework Pytorch to perform a uniform scale transformation on the existing dataset.

[0034] Since the total amount of high-quality labeled data in the dataset is small, the hybrid architecture often cannot reach the point of convergence, so the random data augmentation method is used to expand the original dataset. The quality of data augmentation largely determines the quality of the dataset.

[0035] In this embodiment, a reasonable enhancement method is designed according to the characteristics of the image data set, wherein the enhancement method includes a random combination of random flipping, color jittering, and noise addition, and the change values ​​of the enhancement method all use random numbers within a certain range.

[0036] Each type of labeled data in the dataset is divided into a training set and a test set in a ratio of 8:2. It should be noted that the labels of the expanded labeled dataset and the unlabeled dataset remain unchanged.

[0037] Extracting data features The processed labeled data ;in, and Respectively represent data and their labels, Represents the number of labeled data; the processed labeled data are input into the student network and the teacher network in turn, and first undergo a convolution operation with a convolution kernel size of 1 to preprocess the original features and map them to a larger feature space. After the convolution operation in the local feature extractor within the block, they enter the average pooling layer to extract the average features of the samples and obtain the local features in multiple blocks. Then, the self-attention mechanism in the global feature extractor is used to capture long distances to form new features with global features, in preparation for subsequent noise perception and knowledge decoupling.

[0038] Local feature noise perception The local features extracted by the convolution process in the teacher model are input into the local noise perceptron respectively. In essence, it contains multiple classification heads of different scales. The intermediate features of each sample pass through its classification head to obtain the probability distribution of the sample. The probability with the largest probability value of the sample is taken as the basis for judging the distinctiveness of the local feature. The index position of the maximum probability is compared with the sample label that complies with the "one-hot" encoding. If the comparison is successful, the corresponding position in the noise mark is recorded as True, indicating that this local feature is a discriminative local feature; if the comparison fails, the corresponding position in the noise mark is recorded as False, indicating that the knowledge contains a lot of noise, and redundant local features are identified.

[0039] Global feature noise perception The global features extracted by the Transformer block process in the teacher model are input into the global noise perceptron respectively. In essence, it contains multiple classification heads of different scales. The intermediate features of each sample pass through its classification head to obtain the probability distribution of the sample. The probability with the largest probability value of the sample is taken as the basis for judging the discriminability of the global feature knowledge. The index position of the maximum probability is compared with the sample label that complies with the "one-hot" encoding. If the comparison is successful, the corresponding position in the noise mark is recorded as True, indicating that this global feature knowledge is discriminative global feature knowledge; if the comparison fails, the corresponding position in the noise mark is recorded as False, indicating that the knowledge contains a lot of noise, and redundant global feature knowledge is judged.

[0040] Local feature knowledge decoupling This embodiment decouples the local feature knowledge according to each table item in the noise identification table. There are only two values ​​of True or False in the table items. True represents the discriminative local feature knowledge, and False represents the redundant local feature knowledge.

[0041] Although the noise identification table and the noise sensor cooperate with each other, they actually exist independently and do not require pre-training. They only play a role during distillation. Such a setting does not increase the complexity of the model itself. On the one hand, this strategy allows the original knowledge to be effectively retained, while at the same time achieving refined knowledge decoupling, ensuring that the student model can focus on effective information during the learning process and maximize its performance. Within the local knowledge, two distillation channels are set up to achieve differentiated distillation of discriminative feature knowledge and redundant feature knowledge.

[0042] Global feature knowledge decoupling The global feature knowledge is decoupled according to each entry in the noise identification table. There are only two values ​​in the entry: True or False. True represents discriminative knowledge, and False represents redundant knowledge.

[0043] Although the noise identification table and the noise sensor cooperate with each other, they actually exist independently and do not require pre-training. They only play a role during distillation. This setting does not increase the complexity of the model itself. On the one hand, this strategy allows the original knowledge to be effectively retained, while at the same time achieving refined knowledge decoupling, ensuring that the student model can focus on effective information during the learning process and maximize its performance. Within the global knowledge, two distillation channels are set up to achieve differentiated distillation of discriminative feature knowledge and redundant feature knowledge.

[0044] Adaptive distillation of local feature knowledge Local feature knowledge distillation includes discriminative local feature knowledge distillation and redundant local feature knowledge distillation. For discriminative local feature knowledge distillation, the mean square error loss function is introduced as the discriminative image local feature knowledge distillation loss function of the alignment operation, and the weight of the discriminative image local feature knowledge distillation loss function is set to In addition, for redundant local feature knowledge distillation, the mean absolute error is also introduced as the redundant image local feature knowledge distillation loss function of the redundant feature alignment operation, and the weight of the redundant image local feature knowledge distillation loss function is set to .

[0045] The progressive adaptive weight adjustment strategy adopted in this embodiment is, at the initial stage, As the training progressed, Gradually greater than ;Right now ; ; in, i Indicates the training process i Iteration process of times; Indicates The weight of local discriminative knowledge during round iteration; Indicates The weight of local discriminative knowledge during round iteration; Indicates The weight of local redundant knowledge during round iteration; Indicates The weight of local redundant knowledge during round iteration; Indicates the rate of increase or decrease of weight in the next iteration.

[0046] This setting not only effectively ensures the learning of discriminative knowledge, but also learns sample ambiguity from redundant knowledge to enhance feature discriminability. The total loss of local feature distillation is the sum of the two losses.

[0047] Through continuous back propagation, the loss value is minimized and the learning of local feature knowledge is completed. The specific loss formula is as follows:

[0048]

[0049]

[0050] in, Represents the knowledge distillation loss function of discriminative image local features; Represents the redundant image local feature knowledge distillation loss function; Represents the image local feature knowledge distillation loss function; represents the local features extracted by the teacher network; represents the local features extracted by the student network; Represents the noise identification table, where True represents the discriminative local feature knowledge extracted by the teacher network, and False represents the redundant local feature knowledge extracted by the teacher network; equal The number of True values, equal The number of Boolean values ​​that are False.

[0051] Adaptive distillation of global feature knowledge Global feature knowledge distillation includes discriminative global feature knowledge distillation and redundant global feature knowledge distillation. For discriminative global feature knowledge distillation, The mean square error loss function is introduced as the discriminative image global feature knowledge distillation loss function for the alignment operation, and the weight of the discriminative image global feature knowledge distillation loss function is set to In addition, for redundant global feature knowledge distillation, the mean absolute error is also introduced as the redundant image global feature knowledge distillation loss function of the redundant feature alignment operation, and the weight of the redundant image global feature knowledge distillation loss function is set to .

[0052] The progressive adaptive weight adjustment strategy adopted in this embodiment is, at the initial stage, As the training progressed, Gradually greater than ;Right now ; ; in, i Indicates the training process i Iteration process of times; Indicates The weight of global discriminative knowledge during round iteration; Indicates The weight of global discriminative knowledge during round iteration; Indicates The weight of global redundant knowledge during round iteration; Indicates The weight of global redundant knowledge during round iteration; Indicates the rate of increase or decrease of weight in the next iteration.

[0053] This setting not only effectively ensures the learning of discriminative knowledge, but also learns sample ambiguity from redundant knowledge to enhance feature discriminability. The total loss of global feature knowledge distillation is the sum of the two losses.

[0054] Through continuous back propagation, the loss value is minimized and the learning of global feature knowledge is completed. The specific loss formula is as follows:

[0055]

[0056]

[0057] in, Represents the discriminative image global feature knowledge distillation loss function; Represents the global feature knowledge distillation loss function of redundant images; Represents the image global feature knowledge distillation loss function; represents the global features extracted by the teacher network; represents the global features extracted by the student network, Represents the noise identification table, where True represents the discriminative global feature knowledge extracted by the teacher network, and False represents the redundant global feature knowledge extracted by the teacher network; equal The number of True values, equal The number of Boolean values ​​that are False.

[0058] Output logic distillation The final extracted features are input into the classification head modules of the teacher network and the student network respectively. After the linear layer and the flattening layer, the final output of the final model is standardized using the Softmax function to obtain the probability distribution of each category. At the same time, distillation is performed with the help of the data labels. The cross entropy is used as the loss function of the distillation. Through continuous back propagation, the loss value is minimized to align the probability distribution of the teacher model and the student model. The specific loss is as follows:

[0059] in, represents the logical knowledge distillation loss function; represents the number of samples, Represents the number of categories, represents the sample index, Represents the teacher model Sample categories The probability of Represents the student model Sample categories The probability of Represents the data " coded label.

[0060] Multi-knowledge adaptive collaborative distillation The core task of this stage is to maximize the synergy of multiple knowledge. To this end, this embodiment introduces an innovative "loss-contribution" evaluation mechanism, which dynamically evaluates the contribution of each type of knowledge by back-propagating the gradient of each loss term, and adjusts the weight of the loss term accordingly; it not only enables the student model to accurately learn the essence of different types of knowledge in the teacher model to meet the diverse needs of samples, but also establishes an efficient synergy mechanism between multiple knowledge, giving full play to the complementary advantages between knowledge, thereby significantly improving the performance and generalization ability of the student model.

[0061] The distillation loss is initially defined as the overall distillation loss function is ; Calculate the gradient of each sub-loss term relative to the overall distillation loss function, that is, the gradient of the overall distillation loss function is ;in, Represents the gradient of the loss function for the global feature knowledge distillation of discriminative images; Represents the gradient of the loss function for the global feature knowledge distillation of redundant images; Represents the gradient of the loss function for the knowledge distillation of discriminative image local features; Represents the gradient of the loss function for the distillation of redundant image local feature knowledge; Represents the gradient of the logical knowledge distillation loss function.

[0062] A multi-layer perceptron (MLP) is used to automatically learn the weight of each loss term based on these gradient features. To achieve this goal, the gradients of all sub-loss terms are concatenated into a feature vector as the input of the MLP. At the same time, in order to avoid imbalance caused by too large or too small weights, the weights are normalized to ensure that all loss terms can play their full role.

[0063]

[0064]

[0065] in It is a multi-layer perceptron network, which contains several fully connected layers. The output of the last layer is the weight of each loss term. , whose input is the gradient information .

[0066]

[0067] After this stage of adaptive distillation, the total distillation loss is It can be expressed as:

[0068] in, Represents the weight of the discriminative image global feature knowledge distillation loss function; Represents the weight of the redundant image global feature knowledge distillation loss function; Represents the weight of the discriminative image local feature knowledge distillation loss function; Represents the weight of the redundant image local feature knowledge distillation loss function; Represents the weight of the logical knowledge distillation loss function.

[0069] In the process of training the model, this embodiment uses the back propagation algorithm of the error to calculate the gradient; uses the stochastic gradient descent method to continuously update the parameters of the student network, and selects some model parameters to update along the direction of the inverse gradient in each round of training; iterates continuously until the model performance and the loss value calculated by the loss function stabilize in a smaller range, and stops training when the convergence requirement is reached, and saves the network model at the minimum loss value.

[0070] In this embodiment, the image to be classified is input into the trained network model for prediction. Finally, the category corresponding to the maximum value calculated by the network model output is used as the final prediction result, and the final category is output through key-value mapping.

[0071] This embodiment applies a variety of knowledge to multiple distillation processes, which not only enriches the learning content of the student model, but also greatly taps the utilization potential of feature knowledge; at the same time, a noise sensor is introduced to determine the noise position, ensure the quality of the distilled feature knowledge, and reduce the negative impact of teacher noise on student ability improvement. Through the progressive weight adjustment strategy, it can effectively balance the weight distribution of discriminative knowledge and redundant knowledge, thereby optimizing the distillation process and improving the learning effect of the student model; multivariate knowledge adaptive collaborative distillation satisfies the differentiated needs of samples. It can better integrate multivariate knowledge from different perspectives to ensure that the student model can be optimized in multiple dimensions. This synergy can effectively improve the comprehensive ability of the student model, especially when processing complex and diverse data; adaptive weighting can more flexibly respond to the diverse needs of various samples, reduce redundant interference, and effectively improve the learning ability and generalization performance of the model.

[0072] Embodiment 2 Embodiment 2 of the present invention introduces an image classification system based on two-stage adaptive knowledge distillation.

[0073] like Figure 3 An image classification system based on two-stage adaptive knowledge distillation is shown, comprising: An acquisition module, configured to acquire an image to be classified; An extraction module, which is configured to extract local image features and global image features of the acquired image to be classified based on the student model and the teacher model; A classification module is configured to process the image to be classified according to the acquired local features and global features of the image and the two-stage adaptive knowledge distillation to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

[0074] The detailed steps are the same as those of the image classification method based on two-stage adaptive knowledge distillation provided in Example 1, and will not be repeated here.

[0075] Embodiment 3 Embodiment 3 of the present invention provides a computer-readable storage medium.

[0076] A computer-readable storage medium stores a program thereon, which, when executed by a processor, implements the steps of an image classification method based on two-stage adaptive knowledge distillation as described in Embodiment 1 of the present invention.

[0077] The detailed steps are the same as those of the image classification method based on two-stage adaptive knowledge distillation provided in Example 1, and will not be repeated here.

[0078] Embodiment 4 A fourth embodiment of the present invention provides an electronic device.

[0079] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in an image classification method based on two-stage adaptive knowledge distillation as described in Embodiment 1 of the present invention are implemented.

[0080] The detailed steps are the same as those of the image classification method based on two-stage adaptive knowledge distillation provided in Example 1, and will not be repeated here.

[0081] Embodiment 5 Embodiment 5 of the present invention provides a computer program product.

[0082] A computer program product includes software codes, wherein the program in the software codes executes the steps in an image classification method based on two-stage adaptive knowledge distillation as described in Embodiment 1 of the present invention.

[0083] The detailed steps are the same as those of the image classification method based on two-stage adaptive knowledge distillation provided in Example 1, and will not be repeated here.

[0084] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0085] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0089] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0090] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. An image classification method based on two-stage adaptive knowledge distillation, characterized in that: include: Get the image to be classified; Based on the student model and the teacher model, extract the local image features and the global image features of the acquired image to be classified; According to the acquired local features and global features of the image, as well as the two-stage adaptive knowledge distillation, the image to be classified is processed to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

2. The image classification method based on two-stage adaptive knowledge distillation as claimed in claim 1, characterized in that: The extracted local features and global features of the image are subjected to noise perception and knowledge decoupling respectively to obtain discriminative local features, redundant local features, discriminative global features and redundant global features; the obtained discriminative local features, redundant local features, discriminative global features and redundant global features are subjected to adaptive knowledge distillation to complete the noise-aware adaptive knowledge distillation of local features and global features of the image.

3. The image classification method based on two-stage adaptive knowledge distillation as claimed in claim 2, characterized in that: In the process of adaptive knowledge distillation, mean square error loss function and mean absolute error are introduced for discriminative knowledge and redundant features respectively. Through progressive adaptive weight adjustment, the weight distribution of discriminative knowledge and redundant knowledge is balanced, the adaptive distillation process is optimized, and the discriminative image local feature knowledge distillation loss function, redundant image local feature knowledge distillation loss function, discriminative image global feature knowledge distillation loss function and redundant image global feature knowledge distillation loss function are calculated; logical knowledge distillation is performed on the extracted image local features and image global features respectively, and the logical knowledge distillation loss function is calculated.

4. The image classification method based on two-stage adaptive knowledge distillation as claimed in claim 3, characterized in that: Logical knowledge distillation is performed on the extracted local features and global features of the image respectively, and cross entropy is used as the logical knowledge distillation loss function. Backpropagation is continuously performed. When the logical knowledge distillation loss function is minimized, the probability distributions of the teacher model and the student model are aligned to obtain the logical knowledge distillation loss function.

5. The image classification method based on two-stage adaptive knowledge distillation as claimed in claim 4, characterized in that: Based on the results of the adaptive knowledge distillation and the logical knowledge distillation, multi-knowledge adaptive collaborative distillation is performed; a loss-contribution evaluation mechanism is adopted to dynamically evaluate the contribution of each type of knowledge and adjust its loss function weight by back-propagating the gradient of each distillation loss function.

6. The image classification method based on two-stage adaptive knowledge distillation as claimed in claim 5, characterized in that: According to the obtained discriminative image local feature knowledge distillation loss function, redundant image local feature knowledge distillation loss function, discriminative image global feature knowledge distillation loss function, redundant image global feature knowledge distillation loss function and logical knowledge distillation loss function, the overall distillation loss function and the gradient of each distillation loss function relative to the overall distillation loss function are calculated to obtain the weight of each distillation loss function, and the weighted sum of each distillation loss function is performed to obtain the total distillation loss function, thus completing the two-stage adaptive knowledge distillation.

7. An image classification method system based on two-stage adaptive knowledge distillation, characterized in that: include: An acquisition module, configured to acquire an image to be classified; An extraction module, which is configured to extract local image features and global image features of the acquired image to be classified based on the student model and the teacher model; A classification module is configured to process the image to be classified according to the acquired local features and global features of the image and the two-stage adaptive knowledge distillation to complete the image classification; The two-stage adaptive knowledge distillation includes noise-aware adaptive knowledge distillation and multi-knowledge adaptive collaborative distillation.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of an image classification method based on two-stage adaptive knowledge distillation as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of an image classification method based on two-stage adaptive knowledge distillation are implemented as described in any one of claims 1-6.

10. A computer program product comprising software code, characterized in that The program in the software code executes the steps of an image classification method based on two-stage adaptive knowledge distillation as described in any one of claims 1-6.

Images