Animal image recognition method and device based on knowledge distillation

Abstract

The application discloses an animal image recognition method and device based on knowledge distillation, and the method comprises the following steps: obtaining an animal image to be recognized; inputting the animal image to be recognized into a trained student network, so that the trained student network extracts image information of the animal image to be recognized, and then determines an animal category of the animal image to be recognized according to the extracted image information; and the trained student network is constructed in the following manner: inputting an animal image sample training set into a trained teacher network, so that the trained teacher network outputs a classification probability corresponding to each animal image sample and a temperature coefficient; constructing an initial student network; for each animal image sample, taking the animal image sample, the classification probability corresponding to the animal image sample and the temperature coefficient as input, taking an animal category corresponding to the animal image sample as output, training the initial student network until the loss function of the initial student network is the minimum, and generating the trained student network.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to an animal image recognition method and apparatus based on knowledge distillation. Background Technology

[0002] Knowledge distillation is a model compression method designed to build efficient networks for resource-constrained devices. Knowledge distillation typically involves a student network and a teacher network. Its core idea is to allow a simple student network to learn knowledge from a complex teacher network, thereby enabling the student network to adapt to more real-time application scenarios. Logical knowledge distillation refers to the student network learning from the teacher network only at the output layer. It primarily uses a softmax function with a high-temperature coefficient to generate soft targets, also known as dark knowledge, to represent the class relationships at the teacher network's output layer. The student network learns dark knowledge by minimizing the distance between itself and the soft targets output by the teacher network.

[0003] However, existing logic-based knowledge distillation methods typically use a fixed temperature coefficient to generate soft targets, making it difficult to effectively mine rich dark knowledge from samples of varying difficulty. For example, when animal images are input into a teacher network after using a fixed temperature coefficient in knowledge distillation, and the teacher network outputs the animal names corresponding to the identified images, the use of the same temperature coefficient for all samples can lead to problems. If the animal images to be recognized include both easily identifiable (simple) and difficult (hard) images, an excessively high temperature can over-smooth the knowledge of hard samples, while an excessively low temperature can make the dark knowledge on simple samples almost equivalent to the true labels. Therefore, using the same temperature coefficient for knowledge distillation on all animal image samples during animal image recognition reduces the accuracy of the animal image recognition probability (i.e., the teacher network's dark knowledge) output by the teacher network. Consequently, the knowledge of the teacher network on some animal image samples cannot be effectively transferred to the student network, resulting in a decrease in the recognition accuracy of the ultimately trained student network when recognizing animal images. Summary of the Invention

[0004] This invention provides an animal image recognition method and apparatus based on knowledge distillation, which enables the student network in knowledge distillation to achieve higher accuracy when using the knowledge distillation method for animal image recognition.

[0005] One embodiment of the present invention provides an animal image recognition method based on knowledge distillation, comprising:

[0006] Acquire images of the animal to be identified;

[0007] inputting the animal image to be identified into the trained student network, so that the trained student network extracts image information of the animal image to be identified, and then determines an animal category of the animal image to be identified according to the extracted image information;

[0008] wherein the trained student network is constructed by the following manner:

[0009] obtaining an animal image sample training set and a trained teacher network;

[0010] inputting the animal image sample training set into the trained teacher network, so that the trained teacher network outputs a classification probability corresponding to each animal image sample in the animal image sample training set and a temperature coefficient;

[0011] constructing an initial student network;

[0012] selecting an animal image sample from the animal image sample training set;

[0013] for each animal image sample, taking the animal image sample, the classification probability corresponding to the animal image sample and the temperature coefficient as input, and taking an animal category corresponding to the animal image sample as output, training the initial student network until a loss function of the initial student network is minimized, to generate the trained student network.

[0014] Further, the inputting the animal image sample training set into the trained teacher network, so that the trained teacher network outputs a classification probability corresponding to each animal image sample in the animal image sample training set and a temperature coefficient, comprises:

[0015] obtaining an animal image sample from the animal image sample training set;

[0016] inputting the animal image sample into the trained teacher network one by one, to determine Logits output by each animal image sample in the teacher network;

[0017] determining a classification probability corresponding to each animal image sample according to the Logits output by each animal image sample;

[0018] determining an information entropy of each animal image sample according to the classification probability corresponding to each animal image sample;

[0019] generating a temperature coefficient of each animal image sample according to the information entropy of each animal image sample, a total number of samples of the animal image sample training set, an initial temperature coefficient and a preset function.

[0020] Further, the determining a classification probability corresponding to each animal image sample according to the Logits output by each animal image sample, comprises:

[0021] The classification probability of each animal image sample is calculated using the following formula:

[0022]

[0023] Where K represents the total number of animal categories; t i，k c represents the Logits output of the i-th animal image sample in the k-th class in the trained teacher network; i，k This represents the classification probability of the i-th animal image sample in the k-th category. and c i，k ∈(0,1).

[0024] Further, determining the information entropy of each animal image sample based on the classification probability corresponding to each animal image sample includes:

[0025] The information entropy of each animal image sample is calculated using the following formula:

[0026]

[0027] Among them, S i Let represent the information entropy of the i-th animal image sample.

[0028] Further, the step of generating the temperature coefficient of each animal image sample based on the information entropy of each animal image sample, the total number of samples in the animal image sample training set, the initial temperature coefficient, and the preset function includes:

[0029] The temperature coefficient of each animal image sample is calculated using the following formula:

[0030]

[0031] Where T represents the initial temperature coefficient; f(·) represents the preset function; N represents the total number of samples in the animal image training set; and T0 represents the total number of samples in the training set. i This represents the temperature coefficient of the i-th animal image sample.

[0032] Furthermore, the first distillation loss generated by the initial student network when learning from the trained teacher network is:

[0033]

[0034]

[0035]

[0036] Where, θ t θ represents the network parameters of the trained teacher network. s This represents the network parameters of the initial student network; represents the first distillation loss generated by the initial student network when learning the trained teacher network; t i，k represents the Logits output by the i th animal image sample in the trained teacher network on the k th class; s i，k represents the Logits output by the i th animal image sample in the initial student network on the k th class.

[0037] Further, before selecting the animal image samples from the animal image sample training set, further comprising:

[0038] Obtain a plurality of animal image samples from the animal image sample training set as selected samples;

[0039] Obtain the temperature coefficient corresponding to the selected sample;

[0040] Determine the mixing parameter of each selected sample according to the temperature coefficient corresponding to all selected samples;

[0041] The animal image sample is input into the trained teacher network one by one to determine the Logits output by each animal image sample in the teacher network, comprising:

[0042] The selected sample and the mixing parameter of the selected sample are input into the trained teacher network to determine the mixed Logits output by the selected sample in the teacher network.

[0043] Further, the second distillation loss generated by the initial student network when learning the trained teacher network is:

[0044]

[0045] Wherein, θ t represents the network parameter of the trained teacher network; θ s represents the network parameter of the initial student network; L M (θ t ; θ s ) represents the second distillation loss generated by the initial student network when learning the trained teacher network; represents the mixed Logits output by the i th selected sample in the trained teacher network on the k th class; represents the mixed Logits output by the i th selected sample in the initial student network on the k th class. represents the classification probability output by the trained teacher network in the mixed Logits; represents the classification probability output by the initial student network in the mixed Logits.

[0046] Further, the loss function of the initial student network is:

[0047]

[0048]

[0049]

[0050] wherein, L represents a loss function of the initial student network; represents a predicted value of the i th animal image sample generated by the initial student network on the k th class; y i，k represents a true value of the i th animal image sample on the k th class; L cls (si, yi) represents a classification loss of the initial student network; represents a first distillation loss generated by the initial student network when learning the trained teacher network; L M (θ t ; θ s ) represents a second distillation loss generated by the initial student network when learning the trained teacher network; and alpha, beta and lambda are weights used to balance the three losses.

[0051] On the basis of the above method embodiment, the application correspondingly provides a device embodiment;

[0052] An embodiment of the application correspondingly provides an animal image recognition device based on knowledge distillation, comprising: an image recognition module and a student network construction module.

[0053] The image recognition module is configured to acquire an animal image to be recognized, and input the animal image to be recognized into a trained student network, so that the trained student network extracts image information of the animal image to be recognized, and then determines an animal category of the animal image to be recognized according to the extracted image information.

[0054] The student network construction module is configured to acquire an animal image sample training set and a trained teacher network, input the animal image sample training set into the trained teacher network, so that the trained teacher network outputs a classification probability and a temperature coefficient corresponding to each animal image sample in the animal image sample training set, construct an initial student network, select an animal image sample from the animal image sample training set, for each animal image sample, take the animal image sample, the classification probability and the temperature coefficient corresponding to the animal image sample as input, and take an animal category corresponding to the animal image sample as output, train the initial student network until a loss function of the initial student network is minimized, and generate the trained student network.

[0055] By implementing the application, the following beneficial effects are achieved:

[0056] The application provides an animal image recognition method and device based on knowledge distillation. The method identifies animal image samples through a trained teacher network to determine the classification probability and temperature coefficient corresponding to each animal image sample in animal type recognition. Thus, when training an initial student network, the animal image sample, the classification probability and temperature coefficient corresponding to the animal image sample are taken as the input of the initial student network, the animal category corresponding to the animal image sample is taken as the output of the initial student network, the initial student network is trained until the loss function of the initial student network is minimized, and a trained student network is obtained; then, the trained student network is used for animal image recognition. The temperature parameter required by each sample in knowledge distillation and the classification probability in the trained teacher network are determined through the trained teacher network, and then the temperature parameter and classification probability of the same sample obtained in the trained teacher network are jointly input into the initial student network when the initial student network is trained with the same sample, so that the knowledge in the trained teacher network can be more fully learned when the initial student network is trained, and the finally generated trained student network has better recognition effect, that is, the accuracy of the trained student network in animal image recognition is improved. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 FIG. 1 is a flowchart of an animal image recognition method based on knowledge distillation according to an embodiment of the application.

[0058] Figure 2 FIG. 2 is a logic diagram of a training process of the animal image recognition method based on knowledge distillation according to an embodiment of the application.

[0059] Figure 3 FIG. 3 is a structural diagram of an animal image recognition device based on knowledge distillation according to an embodiment of the application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0061] To better reflect the effects achieved by the application, some terms related to the application are explained as follows:

[0062] (1) Knowledge distillation: Knowledge distillation is a model compression method that aims to build efficient networks for resource-constrained devices. The knowledge distillation method includes a teacher network and a student network, and its core idea is to let the simple student network learn knowledge from the complex teacher network, so that the student network can adapt to more real-time application scenarios.

[0063] Knowledge distillation mainly has two forms: logic-based and feature-based distillation. Logic-based knowledge distillation means that the student network only learns from the teacher network at the output layer, mainly through a high-temperature coefficient Softmax function to produce soft targets (Soft Target), also known as dark knowledge, to express the class relationship information on the output layer of the teacher network. The student network learns dark knowledge by minimizing the distance between the soft target output by the teacher network. In feature-based distillation, the teacher network also transfers the feature knowledge on the intermediate network layer to the student network. For example, there is a method that allows the hidden layer of the student network to predict similar output as the teacher network at the hidden layer, which is one of the earliest methods of using intermediate feature knowledge of the teacher network. The knowledge on the intermediate feature layer has various types, for example, some researchers use attention maps on the intermediate layers of the teacher network and the student network for knowledge distillation, and some researchers use feature distributions on the intermediate layers of the teacher network and the student network for knowledge distillation. In fact, the feature-based distillation method is mainly to mine various types of knowledge on the intermediate network layer of the teacher network, without improving the dark knowledge output by the ordinary knowledge distillation. This means that the feature-based distillation method and the logic-based distillation method are complementary. Generally speaking, the more levels of knowledge the student network learns from the teacher network, the better the performance of the student network will be. Therefore, the feature-based distillation method is usually better than the logic-based distillation method. Without increasing the training time and storage resources, the logic-based distillation can combine various types of knowledge on the intermediate feature layer to further improve the performance of the student network.

[0064] (2) Mixup method: Mixup is a data augmentation method that generates new images and labels by randomly mixing two training samples and their labels in a certain proportion. It can improve network performance because it helps to smooth the output distribution and increase additional training samples.

[0065] As shown in Figure 1 , it is an animal image recognition method based on knowledge distillation provided by an embodiment of the present application, comprising:

[0066] Step S1: obtaining an animal image to be recognized;

[0067] Step S2: input the animal image to be identified into the trained student network, so that the trained student network extracts image information of the animal image to be identified, and then determines the animal category of the animal image to be identified according to the extracted image information; wherein the trained student network is constructed by: obtaining an animal image sample training set and a trained teacher network; inputting the animal image sample training set into the trained teacher network, so that the trained teacher network outputs a classification probability corresponding to each animal image sample in the animal image sample training set and a temperature coefficient; constructing an initial student network; selecting an animal image sample from the animal image sample training set; for each animal image sample, taking the animal image sample, the classification probability corresponding to the animal image sample and the temperature coefficient as input, and taking the animal category corresponding to the animal image sample as output, training the initial student network until the loss function of the initial student network is minimized, and generating the trained student network.

[0068] For step S1, an animal image to be identified is obtained.

[0069] For step S2, the animal image to be identified obtained in step S1 is input into the trained student network for identifying animal images, so that the trained student network extracts image information in the input animal image to be identified, and determines the animal category corresponding to the image according to the extracted image information. For example, an image containing a cat is input into the trained student network, the trained student network extracts image information such as cat ears and cat tail in the image, and determines that the animal category corresponding to the image is a cat according to the extracted image information.

[0070] To improve the accuracy of animal image category identification, the trained student network is constructed by the following method.

[0071] An animal image sample training set containing N animal image samples is obtained wherein y i =N K ={1, 2, …, K}, in the above formula, x i is the i-th animal image sample in the training set Δ, y i is the real category corresponding to the u-th animal image sample, and K is the total number of animal categories.

[0072] In order to enable knowledge distillation using data of the training set to select a suitable temperature coefficient for distillation according to different samples, the present application needs to obtain a trained teacher network to identify the difficulty of data in the training set.

[0073] It should be noted that the trained teacher network is a teacher network capable of evaluating the difficulty level of the input image sample recognition, and a large and deep network model that has performed well in image recognition or object recognition tasks can be used as the teacher network. For example, a deep residual network (ResNet) can be used as the teacher network, which is a network model commonly used in object recognition tasks. It solves the problems of gradient vanishing and degradation in deep network training by introducing residual connection. We can choose different layers of ResNet as the teacher network, such as ResNet-50, ResNet-101 or ResNet-152, etc.

[0074] For the determination of the difficulty level of image sample recognition, it is generally believed that the inter-class relationship of difficult samples is relatively complex, while the inter-class relationship of simple samples is relatively simple. For example, sample a contains 60% cat features and 40% dog features, and sample b contains 90% cat features and 10% dog features. Although sample a and sample b will be accurately classified as cats in the training stage, sample a contains relatively less cat features than sample b, and although sample a and sample b will be accurately classified as cats in the training stage, sample a is obviously more challenging than sample b. The reason behind this is that during the training process of classifying these two samples as cats, the former is more susceptible to interference from dog features than the latter. After training of the two samples, the uncertainty of the output result of the teacher network on sample a is usually higher than that on sample b.

[0075] In the present application, the uncertainty of the output result is used to evaluate the information of the classification result of the trained teacher network for animal classes. If a sample has higher complexity, the output result of the trained teacher network on the sample will receive greater uncertainty. Therefore, the present application uses the uncertainty of the output result of the trained teacher network on all animal image samples in the animal image sample training set to evaluate the relative difficulty level of recognizing an animal image sample.

[0076] In a preferred embodiment, the inputting the animal image sample training set into the trained teacher network to make the trained teacher network output the classification probability and the temperature coefficient corresponding to each animal image sample in the animal image sample training set comprises: obtaining an animal image sample from the animal image sample training set; inputting the animal image sample into the trained teacher network one by one to determine the Logits output by each animal image sample in the teacher network; determining the classification probability corresponding to each animal image sample according to the Logits output by each animal image sample; determining the information entropy of each animal image sample according to the classification probability corresponding to each animal image sample; and generating the temperature coefficient of each animal image sample according to the information entropy of each animal image sample, the total number of samples in the animal image sample training set, an initial temperature coefficient and a preset function.

[0077] Specifically, as shown in FIG. 1, which is a logic diagram of a training process of an animal image recognition method based on knowledge distillation provided by an embodiment of the present application. Figure 2

[0078] The animal image sample training set is obtained, and animal image samples are obtained therefrom one by one and input into a trained teacher network, which contains an already trained parameter θ t When the i th animal image sample is input into the trained teacher network, let the Logits output by the trained teacher network on the i th animal image sample be: represents a set containing K real numbers.

[0079] In a preferred embodiment, the determining the classification probability corresponding to each animal image sample according to the Logits output by each animal image sample comprises:

[0080] The classification probability corresponding to each animal image sample is calculated by the following formula:

[0081]

[0082] wherein K represents the total number of animal categories; t i，k represents the Logits output by the i th animal image sample in the trained teacher network on the k th category; c i，k represents the classification probability of the i th animal image sample in the k th category, and c i，k ∈(0, 1).

[0083] ​Specifically, after setting the Logits output by the trained teacher network on the i-th animal image sample, the classification probability of the i-th animal image sample in the k-th category can be obtained according to the above classification probability calculation formula.

[0084] In a preferred embodiment, the information entropy of each animal image sample is determined according to the classification probability corresponding to each animal image sample, including:

[0085] The information entropy of each animal image sample is calculated by the following formula:

[0086]

[0087] Wherein, S i represents the information entropy of the i-th animal image sample.

[0088] Specifically, after obtaining the classification probability of the i-th animal image sample in the k-th category, the information entropy of the i-th animal image sample can be calculated according to the above formula.

[0089] However, the difficulty of identifying an animal image sample is difficult to accurately evaluate only by its corresponding calculated information entropy. To solve this problem, the present application adopts the idea of "relative difficulty". That is, when the information entropy value of a sample is greater than the information entropy value of other samples, the sample is considered to be more difficult to identify than other samples. The greater the information entropy, the more difficult it is to identify the animal image sample, and then the corresponding temperature coefficient needs to be reduced.

[0090] In a preferred embodiment, the temperature coefficient of each animal image sample is generated according to the information entropy of each animal image sample, the total number of samples of the animal image sample training set, the initial temperature coefficient and the preset function, including:

[0091] The temperature coefficient of each animal image sample is calculated by the following formula:

[0092]

[0093] Wherein, T represents the initial temperature coefficient; f(·) represents the preset function; N represents the total number of samples of the animal image sample training set; T i represents the temperature coefficient of the i-th animal image sample.

[0094] Specifically, in the above formula, the temperature coefficient required by the larger information entropy is higher, so when setting the preset function, the preset function f(·) is set as a decreasing function, and in the present application, the preset function is f(x)=-x. Through this function, the temperature coefficient of the low information entropy sample can be increased, and the temperature coefficient of the high information entropy sample can be reduced. Before knowledge distillation, the present application obtains a temperature coefficient set corresponding to each sample in the training set by performing the above series of processes on the N animal image samples obtained by the trained teacher network: Optionally, to make the temperature in the finally obtained temperature coefficient set one-to-one corresponding to the training set, each sample in the temperature coefficient set is pre-numbered, that is, the temperature coefficient of the i-th sample is T i , x i .

[0095] After determining the temperature coefficient required by each animal image sample in the animal image sample training set when performing knowledge distillation through the trained teacher network, an initial student network is constructed, and knowledge distillation is performed to enable the knowledge learned in the trained teacher network to be migrated to the initial student network. When training the initial student network, the same training set as input into the trained teacher network is used for training. At the same time, the temperature coefficient set obtained by the trained teacher network and the classification probability are used as the input of the initial student network, so that the knowledge learned in the trained teacher network is migrated to the initial student network. During the training of different animal image samples, the corresponding temperature parameter and classification probability are selected from the temperature coefficient set according to the sample number for knowledge distillation, and at the same time, the first distillation loss generated by the initial student network when learning the trained teacher network can be obtained.

[0096] In a preferred embodiment, the first distillation loss generated by the initial student network when learning the trained teacher network is:

[0097]

[0098]

[0099]

[0100] wherein, θ t represents the network parameters of the trained teacher network; θ s represents the network parameters of the initial student network; represents the first distillation loss generated by the initial student network when learning the trained teacher network; t i，k represents the Logits output by the i-th animal image sample in the trained teacher network on the k-th class; s i，krepresents the Logits output by the initial student network on the i-th animal image sample on the k-th class.

[0101] Specifically, represents the dark knowledge output by the trained teacher network on the i-th animal image sample. represents the dark knowledge output by the initial student network on the i-th animal image sample. The first distillation loss generated by the initial student network when learning the trained teacher network is based on the temperature coefficient obtained for each animal image sample. When different temperature parameters are used for knowledge distillation for different samples, the KL divergence-based knowledge distillation loss commonly used at present can be replaced, thereby improving the performance of the student network.

[0102] The present application further improves the network performance of the student network obtained after training, and proposes a new improved method based on the existing Mixup method.

[0103] It should be noted that in the existing research, the idea of Mixup is usually used for data augmentation. Specifically, they randomly mix two training images and the corresponding labels to generate new training samples

[0104]

[0105]

[0106] wherein, lambda is in (0, 1), is a hyperparameter that controls the interpolation strength between the sample pairs.

[0107] The existing Mixup is applied in knowledge distillation, which improves the network performance in knowledge distillation. Since it uses a mixed method to generate new samples, the cardinality of the training samples obtained after using the Mixup method is greatly improved, so that the performance of the network obtained after training is greatly improved. A large part of the reason is due to the data augmentation, that is, the network performance is improved by changing the number of training samples, rather than by using the knowledge distillation method to improve the network performance.

[0108] Therefore, based on the drawbacks of the existing Mixup method in knowledge distillation, the present application proposes a new strong method, which is not used to increase the number of samples, but to increase the richness of the dark knowledge from the samples.

[0109] In a preferred embodiment, before selecting animal image samples from the animal image sample training set, further comprising: obtaining a plurality of animal image samples from the animal image sample training set as selected samples; obtaining the temperature coefficients corresponding to the selected samples; determining the mixing parameters of each selected sample according to the temperature coefficients corresponding to all selected samples; and the inputting of the animal image samples into the trained teacher network one by one to determine the Logits output by each animal image sample in the teacher network, comprising: inputting the selected samples and the mixing parameters of the selected samples into the trained teacher network to determine the mixed Logits output by the selected samples in the teacher network.

[0110] Specifically, before training the initial student network by selecting animal image samples from the animal image sample training set. In each iteration of the trained teacher network, N1 animal image samples are randomly selected from the animal image sample training set Δ (i.e. the selected samples) are used to update the network, and the index numbers of the selected samples are The corresponding temperature coefficients are obtained from the temperature coefficient set according to the index numbers of the selected samples, and the corresponding mixing parameters λ can be obtained by the following formula:

[0111]

[0112] wherein λ represents the mixing parameter; max(·) is the maximum function; T I The temperature coefficients of the N1 selected samples for knowledge distillation are included. The N1 real number sets are included.

[0113] λ is the λ value for the N1 samples, and the calculation result satisfies the existing Mixup method for generating new training samples The requirements for λ in the corresponding formula are:

[0114]

[0115]

[0116] wherein λ∈(0, 1) is a hyperparameter for controlling the interpolation strength between the sample pairs.

[0117] After obtaining the corresponding mixing parameters, the mixing parameters are applied to the output Logits (logic) of the network corresponding to the knowledge distillation of the application.

[0118] The application of the mixing parameters to the knowledge distillation process of the application has the following mathematical representation:

[0119]

[0120]

[0121] where t i and s i represent the Logits output by the teacher network and the student network respectively on the i-th training sample in Δ1. λ i represents the λ value of the i-th sample in Δ1.

[0122] The more difficult a sample is, the higher the λ i value of that sample will be. As can be seen from the above formula, for a relatively simple sample, the present application adds more Logits from other samples to its mixture, while for a relatively difficult sample, the present application adds less Logits from other samples to its mixture. Therefore, the method proposed by the present application can enrich the output knowledge of simple samples, thereby increasing the amount of dark knowledge that the student network can obtain. For example: relatively simple animal image samples such as chickens; relatively difficult animal image samples such as cats or tigers in the cat family. Because chickens in animals are very different from cats / tigers; while cats and tigers are more similar. For a relatively simple sample such as a chicken, we will fuse it with a certain proportion of cat / tiger pictures, and finally we hope to be able to predict the proportion of chicken features and the proportion of cat / tiger in the fused picture.

[0123] The present application mixes the Logits output by the network by using the obtained mixing parameters to obtain mixed Logits. In essence, it makes the output knowledge of simple samples more difficult. For difficult knowledge, we should use low temperature to perform knowledge distillation.

[0124] In a preferred embodiment, the second distillation loss generated by the initial student network when learning the trained teacher network is:

[0125]

[0126] where θ t represents the network parameters of the trained teacher network; θ s represents the network parameters of the initial student network; L M (θ t ; θ s ) represents the second distillation loss generated by the initial student network when learning the trained teacher network; represents the mixed Logits output by the i-th selected sample in the trained teacher network on the k-th class; represents the mixed Logits output by the i-th selected sample in the initial student network on the k-th class; represents the classification probability output by the trained teacher network in the mixed Logits; The classification probability of the initial student network output in the mixed Logits is shown.

[0127] Specifically, in the process of transferring the rich dark knowledge of the trained teacher network to the initial student network, the distillation loss generated when the initial student network learns the dark knowledge from the teacher network with mixed Logits can be obtained by the above formula. Among them,

[0128] In a preferred embodiment, the loss function of the initial student network is:

[0129]

[0130]

[0131]

[0132] Wherein, L represents the loss function of the initial student network; The prediction value of the i-th animal image sample generated by the initial student network in the k-th class is shown. i，k The true value of the i-th animal image sample in the k-th class is shown. cls (s i , y i ) represents the classification loss of the initial student network; The first distillation loss generated by the initial student network when learning the trained teacher network is shown. M (θ t ; θ s ) represents the second distillation loss generated by the initial student network when learning the trained teacher network; α, β and λ are the weights used to balance the three losses.

[0133] Specifically, in the process of training the initial student network, the initial student network is trained by minimizing the loss function of the initial student network, so that when the loss function of the initial student network is minimized, the generated student network has better accuracy when performing animal image recognition.

[0134] The loss function L of the initial student network mainly consists of two parts:

[0135] L = aL cls (s i , y i ) + bL dist (s i , t i )

[0136] Wherein, L cls (s i , yi represents the classification loss of the initial student network; L dist (s i , t i ) represents the distillation loss of the initial student network.

[0137]

[0138]

[0139] wherein y i，k represents the true value of the i-th animal image sample in the k-th category, that is, y i，k is the k-th value in the One-hot Encoding of the true label y i .

[0140] The more similar the prediction probability of the initial student network is to the prediction probability of the trained teacher network, the smaller the corresponding distillation loss is. In the present application, the distillation loss L dist (s i , t i ) is composed of L and L M (θ t ; θ s ), therefore, the loss function of the corresponding initial student network is:

[0141]

[0142] By minimizing the above loss function when training the initial student network, a trained student network that can sufficiently learn the knowledge of the trained teacher network can be obtained. The generated student network has similar classification probability to the classification probability output by the trained teacher network when identifying animal images, so that it also has a higher recognition accuracy when identifying difficult animal images.

[0143] By implementing the present application, the following beneficial effects are achieved:

[0144] 1. The present application uses a trained teacher network to evaluate the difficulty of an animal image sample, without relying on an untrained additional network architecture, thereby reducing the training complexity;

[0145] 2. The present application determines the temperature coefficient required in knowledge distillation by evaluating the difficulty of the animal image sample, which can make the student network more effectively learn the rich dark knowledge in different difficulty samples compared to the existing knowledge distillation method using a fixed temperature coefficient;

[0146] 3. This invention constructs a hybrid parameter and applies it to the selected samples to transform simple samples into difficult samples, enabling the student network to learn richer hidden knowledge during the knowledge distillation process, thereby improving the performance of the student network.

[0147] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0148] like Figure 3 As shown, one embodiment of the present invention provides an animal image recognition device based on knowledge distillation, including: an image recognition module and a student network construction module;

[0149] The image recognition module is used to acquire an image of an animal to be identified; input the image of the animal to be identified into a trained student network so that the trained student network can extract image information of the image of the animal to be identified, and then determine the animal category of the image of the animal to be identified based on the extracted image information;

[0150] The student network construction module is used to acquire an animal image sample training set and a trained teacher network; input the animal image sample training set into the trained teacher network so that the trained teacher network outputs the classification probability and temperature coefficient corresponding to each animal image sample in the animal image sample training set; construct an initial student network; select animal image samples from the animal image sample training set; for each animal image sample, use the animal image sample, the corresponding classification probability and temperature coefficient as input, and the corresponding animal category as output to train the initial student network until the loss function of the initial student network is minimized, and then generate the trained student network.

[0151] It should be noted that the device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0152] Those skilled in the art will clearly understand that, for convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0153] The above is the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements are also considered to be within the scope of the present application.

Claims

1. An animal image recognition method based on knowledge distillation, characterized in that, include: Acquire images of the animal to be identified; The image of the animal to be identified is input into a trained student network so that the trained student network can extract image information from the image of the animal to be identified, and then determine the animal category of the image of the animal to be identified based on the extracted image information; The trained student network is constructed using the following methods: Obtain a training set of animal image samples and a trained teacher network; The training set of animal image samples is input into the trained teacher network so that the trained teacher network outputs the classification probability and temperature coefficient corresponding to each animal image sample in the training set. Construct the initial student network; Animal image samples were selected from the animal image sample training set; For each animal image sample, the initial student network is trained by taking the animal image sample, the classification probability and temperature coefficient corresponding to the animal image sample as inputs and the animal category corresponding to the animal image sample as outputs, until the loss function of the initial student network is minimized, and the trained student network is generated. The information entropy of each animal image sample is determined based on its classification probability, including: The information entropy of each animal image sample is calculated using the following formula: in, Indicates the first Information entropy of a single animal image sample; Indicates the first The animal image sample in the first The classification probability in each category and ; Indicates the total number of animal categories; The process of generating the temperature coefficient of each animal image sample based on the information entropy of each animal image sample, the total number of samples in the animal image sample training set, the initial temperature coefficient, and a preset function includes: The temperature coefficient of each animal image sample is calculated using the following formula: in, Indicates the initial temperature coefficient; Indicates a preset function; This represents the total number of samples in the animal image training set; Indicates the first Temperature coefficient of an animal image sample.

2. The animal image recognition method based on knowledge distillation as described in claim 1, characterized in that, The step of inputting the animal image sample training set into the trained teacher network, so that the trained teacher network outputs the classification probability and temperature coefficient corresponding to each animal image sample in the animal image sample training set, includes: Animal image samples are obtained from the animal image sample training set; Animal image samples are input one by one into the trained teacher network to determine the Logits output for each animal image sample in the teacher network; The classification probability of each animal image sample is determined based on the Logits output for each animal image sample. The information entropy of each animal image sample is determined based on the classification probability corresponding to each animal image sample. The temperature coefficient of each animal image sample is generated based on the information entropy of each animal image sample, the total number of samples in the animal image sample training set, the initial temperature coefficient, and a preset function.

3. The animal image recognition method based on knowledge distillation as described in claim 2, characterized in that, The step of determining the classification probability corresponding to each animal image sample based on the Logits output for each animal image sample includes: The classification probability of each animal image sample is calculated using the following formula: in, This indicates the number of trained teachers in the network. The animal image sample in the first Logits output for each category.

4. The animal image recognition method based on knowledge distillation as described in claim 3, characterized in that, The first distillation loss produced by the initial student network when learning from the trained teacher network is: in, Represents the network parameters of the trained teacher network; This represents the network parameters of the initial student network; This represents the first distillation loss produced by the initial student network when learning from the trained teacher network; This indicates the number of trained teachers in the network. The animal image sample in the first Logits output for each category; Indicates the first student in the initial student network The animal image sample in the first Logits output for each category.

5. The animal image recognition method based on knowledge distillation as described in claim 4, characterized in that, Before selecting animal image samples from the animal image sample training set, the following steps are also included: Several animal image samples were selected from the animal image sample training set; Obtain the temperature coefficient corresponding to the selected sample; The mixing parameters for each selected sample are determined based on the temperature coefficients corresponding to all selected samples. The step of inputting animal image samples one by one into the trained teacher network to determine the Logits output for each animal image sample in the teacher network includes: The selected sample and its mixed parameters are input into the trained teacher network to determine the mixed Logits output by the selected sample in the teacher network.

6. The animal image recognition method based on knowledge distillation as described in claim 5, characterized in that, The second distillation loss generated by the initial student network when learning from the trained teacher network is: in, Represents the network parameters of the trained teacher network; This represents the network parameters of the initial student network; This represents the second distillation loss generated by the initial student network when learning from the trained teacher network; This indicates the number of trained teachers in the network. The selected samples in the first Mixed Logits output across each category; Indicates the first student in the initial student network The selected samples in the first Mixed Logits output across each category; This represents the classification probability output by the teacher network trained in the mixed Logits; This represents the classification probability of the initial student network output in the mixed Logits.

7. The animal image recognition method based on knowledge distillation as described in claim 6, characterized in that, The loss function of the initial student network is: in, The loss function represents the initial student network; This represents the first student network generated. The animal image sample in the first Predicted values ​​for each category; Indicates the first The animal image sample in the first The true value for each category; This represents the classification loss of the initial student network; This represents the first distillation loss produced by the initial student network when learning from the trained teacher network; This represents the second distillation loss generated by the initial student network when learning from the trained teacher network; , and These are the weights used to balance these three losses.

8. An animal image recognition device based on knowledge distillation, characterized in that, include: Image recognition module and student network construction module; The image recognition module is used to acquire an image of an animal to be identified; input the image of the animal to be identified into a trained student network so that the trained student network can extract image information of the image of the animal to be identified, and then determine the animal category of the image of the animal to be identified based on the extracted image information; The student network construction module is used to acquire an animal image sample training set and a trained teacher network; input the animal image sample training set into the trained teacher network so that the trained teacher network outputs the classification probability and temperature coefficient corresponding to each animal image sample in the animal image sample training set; construct an initial student network; select animal image samples from the animal image sample training set; for each animal image sample, use the animal image sample, the corresponding classification probability and temperature coefficient as input, and the corresponding animal category as output to train the initial student network until the loss function of the initial student network is minimized, and then generate the trained student network. The information entropy of each animal image sample is determined based on its classification probability, including: The information entropy of each animal image sample is calculated using the following formula: in, Indicates the first Information entropy of a single animal image sample; Indicates the first The animal image sample in the first The classification probability in each category and ; Indicates the total number of animal categories; The process of generating the temperature coefficient of each animal image sample based on the information entropy of each animal image sample, the total number of samples in the animal image sample training set, the initial temperature coefficient, and a preset function includes: The temperature coefficient of each animal image sample is calculated using the following formula: in, Indicates the initial temperature coefficient; Indicates a preset function; This represents the total number of samples in the animal image training set; Indicates the first Temperature coefficient of an animal image sample.

Images