Model training method and device, storage medium and program product

Abstract

The embodiment of the invention provides a model training method and device, a storage medium and a program product, and the method comprises the steps: obtaining a training sample, inputting the training sample into a student model and a teacher model, obtaining a first output value outputted by the student model and a second output value outputted by the teacher model, determining a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value and a truth value label of the training sample, and determining a total loss function according to the distillation loss function and the weight coefficient of the distillation loss function, and training the student model according to the total loss function. According to the model training method provided by the embodiment of the invention, wrong knowledge can be prevented from being introduced, and convergence of student models is accelerated.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer and network communication technology, and in particular, to a model training method, device, storage medium, and program product. Background Art

[0002] Model distillation refers to the use of a higher-precision teacher model (larger parameters) to guide the training of a lower-precision student model (smaller parameters and faster prediction speed) in deep learning model training, thereby improving the accuracy of the small model while ensuring the inference speed.

[0003] In related technologies, the student model is usually trained based on the total loss function obtained by the weighted sum of the true value loss function and the distillation loss function.

[0004] However, the inventors found that there are at least the following technical problems in the related technology: in the calculation process of the total loss function, the weights of the true value loss function and the distillation loss function are fixed, which can easily introduce incorrect guidance and lead to poor model convergence. Summary of the invention

[0005] The disclosed embodiments provide a model training method, device, storage medium, and program product to avoid introducing incorrect guidance to the student model and accelerate model convergence.

[0006] In a first aspect, an embodiment of the present disclosure provides a model training method, comprising:

[0007] Obtain training samples, input the training samples into a student model and a teacher model respectively, and obtain a first output value output by the student model and a second output value output by the teacher model;

[0008] Determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value, and the true value label of the training sample;

[0009] A total loss function is determined according to the distillation loss function and a weight coefficient of the distillation loss function, and the student model is trained according to the total loss function.

[0010] In a second aspect, the present disclosure provides an image processing method, including:

[0011] Get the image to be processed;

[0012] The image to be processed is input into a deep learning model to obtain a prediction result corresponding to the image to be processed; the deep learning model is trained according to the model training method described in the first aspect.

[0013] In a third aspect, an embodiment of the present disclosure provides a model training device, including:

[0014] An input module is used to obtain training samples, input the training samples into the student model and the teacher model respectively, and obtain a first output value output by the student model and a second output value output by the teacher model;

[0015] A determination module, configured to determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value, and the true value label of the training sample;

[0016] A training module is used to determine a total loss function according to the distillation loss function and a weight coefficient of the distillation loss function, and to train the student model according to the total loss function.

[0017] In a fourth aspect, an embodiment of the present disclosure provides an image processing device, including:

[0018] An acquisition module, used for acquiring an image to be processed;

[0019] A prediction module is used to input the image to be processed into a deep learning model to obtain a prediction result corresponding to the image to be processed; the deep learning model is trained according to the model training method according to any one of claims 1 to 8.

[0020] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0021] The memory stores computer-executable instructions;

[0022] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the model training method described in the first aspect and various possible designs of the first aspect, and executes the image processing method described in the second aspect and various possible designs of the second aspect.

[0023] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the model training method described in the first aspect and various possible designs of the first aspect is implemented, and the image processing method described in the second aspect and various possible designs of the second aspect is executed.

[0024] In the seventh aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the model training method described in the first aspect and various possible designs of the first aspect, and executes the image processing method described in the second aspect and various possible designs of the second aspect.

[0025] The model training method, device, storage medium and program product provided in the present embodiment include obtaining training samples, inputting the training samples into a student model and a teacher model respectively, obtaining a first output value output by the student model and a second output value output by the teacher model, determining a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value and the true value label of the training sample, determining a total loss function according to the distillation loss function and the weight coefficient of the distillation loss function, and training the student model according to the total loss function. The model training method provided in the present embodiment of the disclosure can adjust the influence of the distillation loss function of different samples on the gradient by setting a weight coefficient for the distillation loss function based on the output values ​​and true value labels of the student model and the teacher model, so as to adaptively adjust the guiding role of the teacher model for different samples, avoid introducing erroneous knowledge, and accelerate the convergence of the student model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 A schematic diagram of the principle of the model training method provided in the embodiment of the present disclosure;

[0028] Figure 2 A schematic diagram of a model training method flow chart provided in an embodiment of the present disclosure;

[0029] Figure 3 A schematic diagram of the image processing method provided by the embodiment of the present disclosure;

[0030] Figure 4 A structural block diagram of a model training device provided in an embodiment of the present disclosure;

[0031] Figure 5 A structural block diagram of an image processing device provided by an embodiment of the present disclosure;

[0032] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0034] In deep learning tasks, models with large model parameters tend to have high accuracy but slow prediction speed, while models with small model parameters tend to have fast prediction speed but low accuracy. Model distillation refers to using a high-precision teacher model (larger parameters) to guide the training of a low-precision student model (smaller parameters, faster prediction speed), thereby improving the accuracy of the small model while ensuring the inference speed.

[0035] According to the task, model distillation can be divided into distillation for classification tasks and distillation for regression tasks. For classification tasks, the loss function of model distillation is generally designed to learn probability distribution, that is, let the student model learn the output distribution information of the teacher model; while for regression tasks, the loss function of model distillation is generally designed to learn distance, that is, let the student model learn the output results of the teacher model and calculate the absolute deviation.

[0036] In related technologies, the student model is usually trained using a total loss function obtained by weighting the true value loss function (representing the loss value between the student model output and the true value label) and the distillation loss function (representing the loss value between the student model output and the teacher model output).

[0037] However, in the above method, during the calculation of the total loss function, the weights of the true value loss function and the distillation loss function are fixed, which can easily introduce incorrect guidance and lead to poor model convergence.

[0038] In order to solve the above technical problems, the inventors of this application have found that the reference value of the output of the teacher model is different for different training samples. If the guidance of the teacher model is introduced in the same way for different training samples, erroneous knowledge will be introduced into the student model, resulting in slow model convergence. For example, for training samples where the student model performs well but the teacher model performs poorly, if the guidance of the teacher model is still followed and the distillation loss function determined by the output of the teacher model and the output of the student model is added, then it is equivalent to introducing erroneous knowledge into the student model. Therefore, the inventors have found that the training samples can be distinguished, and the corresponding weight coefficients for the distillation loss function can be determined for different training samples to adaptively adjust the guidance of the teacher model, avoid introducing erroneous knowledge, and accelerate convergence. Based on this, the present application provides a model training method.

[0039] refer to Figure 1 , Figure 1 The schematic diagram of the principle of the model training method provided in the embodiment of the present disclosure is as follows. Figure 1 As shown, the training sample set includes multiple samples, each sample has a true value label. During the training process, in each iteration round, a sample to be trained is selected from the training sample set. Taking the sample to be trained as a single sample as an example, the training sample is input into the student model and the teacher model respectively to obtain the first output value output by the student model and the second output value output by the teacher model. The distillation loss function and the weight coefficient of the distillation loss function are determined according to the first output value, the second output value and the true value label of the training sample. According to the distillation loss function and the weight coefficient of the distillation loss function, the total loss function is determined, and the student model is trained according to the total loss function. In the embodiment of the present disclosure, by setting the weight coefficient for the distillation loss function based on the output value and the true value label of the student model and the teacher model, the influence of the distillation loss function of different samples on the gradient can be adjusted, so as to adaptively adjust the guiding role of the teacher model for different samples, avoid introducing erroneous knowledge, and accelerate the convergence of the student model. Optionally, the first true value loss function can be determined according to the first output value and the true value label, and then the total loss function can be determined according to the first true value loss function, the distillation loss function and the weight coefficient of the distillation loss function.

[0040] refer to Figure 2 , Figure 2 A flow chart of a model training method provided in an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the model training method includes:

[0041] 201. Obtain training samples, input the training samples into a student model and a teacher model respectively, and obtain a first output value output by the student model and a second output value output by the teacher model.

[0042] In this embodiment, the training sample may include one or more samples, which may be image samples. The teacher model and the student model may both be deep learning models, the teacher model has more parameters than the student model, and the accuracy is higher than the accuracy of the student model. The teacher model and the trained student model can be applied to regression tasks and classification tasks, such as target detection, time series prediction, posture estimation, etc.

[0043] Exemplarily, taking the training sample including a sample as an example, the sample can be input into the student model to obtain a first output value, and the sample can be input into the teacher model to obtain a second output value.

[0044] 202. Determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value, and the true value label of the training sample.

[0045] Specifically, the distillation loss function can be determined according to the first output value and the second output value. The distillation loss function represents the loss between the output of the teacher model and the output of the student model. The weight coefficient of the distillation loss function is used to represent the magnitude of the guiding role of the teacher model on the student model during the training process. Therefore, after making a comprehensive judgment by combining the first output value, the second output value and the true value label corresponding to the input training sample, the guiding role of the distillation loss function corresponding to the training sample on the student model can be increased or decreased by adjusting the size of the weight coefficient of the distillation loss function.

[0046] It should be noted that the weight coefficient here is not related to the weight when the distillation loss function and the true value loss function are weighted together when calculating the total loss function.

[0047] The true value label can be the true value that is pre-labeled for the training sample.

[0048] In one embodiment of the present disclosure, the weight coefficient of the distillation loss function can be determined based on the true value loss function corresponding to the student model and the true value loss function corresponding to the teacher model. It is also possible to determine the consistency of the directions of the two vectors based on a first vector composed of a first output value and a true value label and a second vector composed of a second output value and a true value label, and determine the weight coefficient of the distillation loss function based on the consistency. The above two methods can also be considered comprehensively.

[0049] In one implementable manner, determining the weight coefficient of the distillation loss function based on the first output value, the second output value and the true value label of the training sample can include: determining a first vector based on the first output value and the true value label of the training sample, and determining a second vector based on the second output value and the true value label of the training sample; determining cosine similarity based on the first vector and the second vector, and determining the weight coefficient of the distillation loss function based on the cosine similarity.

[0050] Exemplarily, the value range of cosine similarity is greater than or equal to -1 and less than or equal to 1. When the first vector and the second vector are in opposite directions, the angle is 180 degrees, and the cosine similarity is -1. When the first vector and the second vector are in the same direction, the angle is 0 degrees, and the cosine similarity is 1. The weaker the consistency between the first vector and the second vector, that is, the larger the angle between the two vectors, the problem with the teacher model or the true value label is indicated. In order not to introduce unstable variables, the weight coefficient can be set to a lower value, for example, it can be set to the sum of the cosine similarity and 1, and decreases to 0 as the cosine similarity decreases.

[0051] In another implementable manner, determining the weight coefficient of the distillation loss function based on the first output value, the second output value, and the true value label of the training sample may include: determining the first truth loss function corresponding to the student model and the second truth loss function corresponding to the teacher model based on the first output value, the second output value, and the true value label of the training sample; determining the weight coefficient of the distillation loss function based on the first truth loss function and the second truth loss function.

[0052] Specifically, after inputting the training sample into the student model and the teacher model, and obtaining the first output value of the student model and the second output value of the teacher model, the truth loss function corresponding to the student model and the teacher model can be calculated based on the truth label of the training sample. The first truth loss function of the student model indicates the distance between the first output value of the student model and the truth label, and the second truth loss function of the teacher model indicates the distance between the second output value of the teacher model and the truth label. When the first truth loss function is greater than the second truth loss function, it indicates that the output of the teacher model is closer to the truth label, the teacher model has higher accuracy, and the teacher model has a certain guiding role, so the weight coefficient of the distillation loss function can be set to a larger value, such as greater than or equal to 1. When the second truth loss function is less than or equal to the second truth loss function, it indicates that the output of the student model is closer to the truth label, the student model has higher accuracy, and the teacher model has an opposite guiding role, so the weight coefficient of the distillation loss function can be set to a smaller value, such as greater than or equal to 0 and less than 1.

[0053] Exemplarily, the loss function between the student model and the true value label gti can be calculated based on the following expression (1), and the loss function between the teacher model and the true value label gti can be calculated based on expression (2).

[0054] L si =F(O si ,gt i ) (1)

[0055] L ti =F(O ti ,gt i ) (2)

[0056] Among them, F(,) is the loss function of the student model and the teacher model, Lsi is the loss function of the student model, and Lti is the loss function of the teacher model.

[0057] In one embodiment of the present disclosure, determining the weight coefficient of the distillation loss function according to the first truth loss function and the second truth loss function may include: if the first truth loss function is less than or equal to the second truth loss function, determining the weight coefficient of the distillation loss function as a first preset value; the first preset value is less than 1 and greater than or equal to 0.

[0058] Specifically, when the second truth loss function is less than or equal to the second truth loss function, it indicates that the output of the student model is closer to the true value label, the student model has higher accuracy, and the teacher model has the opposite guiding role, so the weight coefficient of the distillation loss function can be set to a smaller value, such as greater than or equal to 0 and less than 1. Exemplarily, the first preset value can be set to 0, that is, the guiding role of the teacher model can be completely eliminated.

[0059] In one embodiment of the present disclosure, determining the weight coefficient of the distillation loss function according to the first truth loss function and the second truth loss function may include: if the first truth loss function is greater than the second truth loss function, determining the cosine similarity between the first vector and the second vector, and determining the weight coefficient of the distillation loss function according to the cosine similarity; the first vector is determined according to the first output value and the truth label, and the second vector is determined according to the second output value and the truth label.

[0060] Specifically, when the first truth loss function is greater than the second truth loss function, it indicates that the output of the teacher model is closer to the true value label, the teacher model has higher accuracy, and the teacher model has a certain guiding role, so the weight coefficient of the distillation loss function can be set to a larger value, for example, greater than or equal to 1. Considering the model ambiguity caused by the inconsistent direction between the first vector and the second vector, in order to avoid ambiguity, the weight coefficient can be set to be related to the cosine similarity between the first vector and the second vector, and decreases as the angle between the first vector and the second vector increases.

[0061] In one embodiment of the present disclosure, in order to avoid the possibility that the convergence process may be oscillated due to the excessive weight coefficient of the distillation loss function, the weight coefficient is limited. The weight coefficient of the distillation loss function is determined according to the cosine similarity, which may include: if the cosine similarity is less than or equal to 0, then the sum of the cosine similarity and a second preset value is determined as the weight coefficient of the distillation loss function; the second preset value is greater than or equal to 1.

[0062] Exemplarily, if the first truth loss function is greater than the second truth loss function, the cosine similarity may be calculated based on expression (3).

[0063] cs i= cos_sim(O si -gt i , O ti -gt i ) (3)

[0064] Among them, CS i is the first vector O si -gt i and the second vector O ti -gt i The cosine similarity between si is the first output value of the student model, gt i is the true value label of the training sample, O ti is the second output value of the teacher model.

[0065] If cs i ≤0, then set w i =1+cs i Among them, w i is the weight coefficient of the distillation loss function.

[0066] In one embodiment of the present disclosure, in order to avoid oscillation in the convergence process caused by an excessively large weight coefficient of the distillation loss function, the weight coefficient is limited. The weight coefficient of the distillation loss function is determined according to the cosine similarity, which may include: if the cosine similarity is greater than 0, the weight coefficient of the distillation loss function is determined to be a third preset value; the third preset value is greater than or equal to 1 and less than 2.

[0067] Exemplarily, the maximum limit of the weight coefficient of the distillation loss function can be set to 1, that is, when the first truth loss function is greater than the second truth loss function and the cosine similarity between the first vector and the second vector is greater than 0, the weight coefficient of the distillation loss function is set to 1.

[0068] 203. Determine a total loss function according to the distillation loss function and a weight coefficient of the distillation loss function, and train the student model according to the total loss function.

[0069] Specifically, after determining the distillation loss function and the weight coefficient of the distillation loss function, the distillation loss function and the weight coefficient of the distillation loss function can be multiplied to obtain an adjusted distillation loss function. Then, the adjusted distillation loss function and the first truth loss function can be weighted and summed to obtain a total loss function. Then, the student model is trained based on the total loss function.

[0070] In one embodiment of the present disclosure, determining the total loss function based on the distillation loss function and the weight coefficient of the distillation loss function may include: determining the product of the distillation loss function and the weight coefficient of the distillation loss function; and performing a weighted sum of the product and the first true value loss function to obtain the total loss function.

[0071] From the above description, it can be seen that this embodiment sets the weight coefficient for the distillation loss function based on the output values ​​and true value labels of the student model and the teacher model, so as to adjust the influence of the distillation loss function of different samples on the gradient, so as to adaptively adjust the guiding role of the teacher model for different samples, avoid the introduction of erroneous knowledge, and accelerate the convergence of the student model.

[0072] In one embodiment of the present disclosure, in the above Figure 2 Based on the embodiment, in order to speed up the training process, the training samples can be input in batches, each batch of training samples includes multiple samples, and the multiple samples are input into the student model and the teacher model at the same time. For this situation, an example is introduced below.

[0073] The training samples include multiple target samples; the first output value includes first target output values ​​corresponding to the multiple target samples respectively, and the second output value includes second target output values ​​corresponding to the multiple target samples respectively.

[0074] In an implementable manner, the total loss function of each sample can be first calculated, and then the average value of the total loss functions of multiple samples can be calculated to obtain the total loss function corresponding to the batch. Specifically, the weight coefficient of the distillation loss function determined according to the first output value, the second output value and the true value label of the training sample can include: for each target sample, according to the first target output value and the second target output value and the target true value label corresponding to the target sample, the first target true value loss function corresponding to the student model and the second target true value loss function corresponding to the teacher model are determined; according to the first target true value loss function and the second target true value loss function, the target weight coefficient of the target distillation loss function corresponding to the target sample is determined; accordingly, according to the distillation loss function and the weight coefficient of the distillation loss function, the total loss function is determined, which can include: for each target sample, according to the first target true value loss function, the target distillation loss function and the target weight coefficient corresponding to the target sample, the target total loss function corresponding to the target sample is determined; the average value of the target total loss functions corresponding to multiple target samples is determined as the total loss function.

[0075] In another possible implementation, the average distillation loss function and the average true value loss function of multiple samples can be calculated separately, and then the weighted sum of the two can be obtained to obtain the total loss function corresponding to the batch. Specifically, determining the weight coefficient of the distillation loss function according to the first output value, the second output value and the true value label of the training sample may include: for each target sample, determining the first target truth loss function corresponding to the student model and the second target truth loss function corresponding to the teacher model according to the first target output value and the second target output value and the target truth label corresponding to the target sample, and determining the target weight coefficient of the target distillation loss function corresponding to the target sample according to the first target truth loss function and the second target truth loss function; accordingly, determining the total loss function according to the distillation loss function and the weight coefficient of the distillation loss function may include: determining the average value of the first target truth loss functions corresponding to multiple target samples as the average truth loss function corresponding to the training sample; determining the average value of the product of the target distillation loss functions and the target weight coefficients corresponding to multiple target samples as the average distillation loss function corresponding to the training sample; and determining the weighted sum of the average truth loss function and the average distillation loss function as the total loss function.

[0076] For example, taking the regression task as an example, during the training process, suppose there are N samples in a batch, and suppose the distillation loss function weight of the i-th sample in the N samples is w i Assuming that the true value label of the sample is gt, first calculate the output O of the student model S si With the output O of the teacher model T ti .

[0077] First, we can calculate the true value label gt of the student model, teacher model and the i-th sample i The loss function L between si and L ti , and calculate the distillation loss function L for the i-th sample di , where L di The expression of can refer to the following formula (4).

[0078] L di =F(O si , O ti ) (4)

[0079] Among them, F(,) is the loss function of the regression task, O si is the first output value of the student model after the i-th sample is input into the student model, O ti is the second output value of the teacher model after the i-th sample is input into the teacher model.

[0080] Secondly, the distillation loss function L of the i-th sample can be calculated di The weight coefficient w i .

[0081] In the first case, if L si Less than or equal to L ti , then set w i =0, indicating that for the i-th sample, the student model has a higher accuracy and the teacher model’s guidance is incorrect, so the distillation loss function is not used.

[0082] In the second case, if the first case does not hold, the cosine similarity CS between the first vector and the second vector i , where the first vector is the first output O of the student model si With the true value label gt i The second vector is the second output O of the teacher model. ti With the true value label gt i If CS i If it is less than or equal to 0, then set w i 1 and CS i The sum between.

[0083] In the third case, if both the first and second cases are not true, then set w i is 1.

[0084] Again, after calculating the distillation loss function of the N samples in the batch and the weight coefficient of the distillation loss function, the average distillation loss function of the batch is calculated, that is, the average value of the product of the distillation loss function of the N samples and the weight coefficient of the distillation loss function.

[0085] Finally, the average value of the true value loss function of the N samples in the batch can be calculated, and then the average value of the true value loss function and the average distillation loss function calculated in the previous step can be weighted summed to obtain the total loss function L of the N samples in the batch. all , and then based on L all Train the student model.

[0086] From the above description, it can be seen that by setting the weight coefficient for the distillation loss function based on the output values ​​of the student model and the teacher model and the true value label, it is possible to adjust the impact of the distillation loss function of different samples on the gradient, so as to adaptively adjust the guidance role of the teacher model for different samples, avoid introducing erroneous knowledge, and accelerate the convergence of the student model.

[0087] refer to Figure 3 , Figure 3A flowchart of an image processing method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the image processing method includes:

[0088] 301. Obtain an image to be processed.

[0089] 302. Input the image to be processed into a deep learning model to obtain a prediction result corresponding to the image to be processed; the deep learning model is obtained by training according to the model training method described in the above embodiment.

[0090] Specifically, in the above embodiments, for example Figure 2 After the model training method described in the illustrated embodiment trains the student model to obtain a deep learning model, the deep model can be set on a terminal device (mobile phone, tablet, computer, etc.), and the image to be processed can be input into the deep learning model to perform classification tasks or regression tasks, such as target detection, time series prediction, posture estimation, etc.

[0091] From the above description, it can be seen that by setting the weight coefficient for the distillation loss function based on the output values ​​of the student model and the teacher model and the true value label, it is possible to adjust the impact of the distillation loss function of different samples on the gradient, so as to adaptively adjust the guidance role of the teacher model for different samples, avoid introducing erroneous knowledge, and accelerate the convergence of the student model.

[0092] Corresponding to the model training method in the above embodiment, Figure 4 This is a structural block diagram of a model training device provided in an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 4 The device includes: an input module 401, a determination module 402 and a training module 403.

[0093] Among them, the input module 401 is used to obtain training samples, input the training samples into the student model and the teacher model respectively, and obtain the first output value output by the student model and the second output value output by the teacher model.

[0094] The determination module 402 is used to determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value and the true value label of the training sample.

[0095] The training module 403 is used to determine the total loss function according to the distillation loss function and the weight coefficient of the distillation loss function, and train the student model according to the total loss function.

[0096] In one embodiment of the present disclosure, the determination module 402 is specifically used to: determine the first truth loss function corresponding to the student model and the second truth loss function corresponding to the teacher model according to the first output value, the second output value and the true value label of the training sample; determine the weight coefficient of the distillation loss function according to the first truth loss function and the second truth loss function.

[0097] In one embodiment of the present disclosure, the determination module 402 is specifically used to: if the first truth loss function is less than or equal to the second truth loss function, determine the weight coefficient of the distillation loss function as a first preset value; the first preset value is less than 1 and greater than or equal to 0.

[0098] In one embodiment of the present disclosure, the determination module 402 is specifically used to: if the first truth loss function is greater than the second truth loss function, determine the cosine similarity between the first vector and the second vector, and determine the weight coefficient of the distillation loss function according to the cosine similarity; the first vector is determined according to the first output value and the truth label, and the second vector is determined according to the second output value and the truth label.

[0099] In one embodiment of the present disclosure, the determination module 402 is specifically used to: if the cosine similarity is less than or equal to 0, determine the sum of the cosine similarity and a second preset value as the weight coefficient of the distillation loss function; the second preset value is greater than or equal to 1.

[0100] In one embodiment of the present disclosure, the determination module 402 is specifically used to: if the cosine similarity is greater than 0, determine the weight coefficient of the distillation loss function as a third preset value; the third preset value is greater than or equal to 1 and less than 2.

[0101] In one embodiment of the present disclosure, the training module 403 is specifically used to: determine the product of the distillation loss function and the weight coefficient of the distillation loss function; and perform a weighted sum of the product and the first true value loss function to obtain a total loss function.

[0102] In one embodiment of the present disclosure, the training sample includes multiple target samples; the first output value includes first target output values ​​corresponding to the multiple target samples respectively, and the second output value includes second target output values ​​corresponding to the multiple target samples respectively; the determination module 402 is specifically used to: for each target sample, determine the first target truth loss function corresponding to the student model and the second target truth loss function corresponding to the teacher model according to the first target output value and the second target output value and the target truth value label corresponding to the target sample, and determine the target weight coefficient of the target distillation loss function corresponding to the target sample according to the first target truth loss function and the second target truth loss function.

[0103] The training module 403 is specifically used to: determine the average value of the first target truth loss function corresponding to the multiple target samples as the average truth loss function corresponding to the training samples; determine the average value of the product of the target distillation loss function and the target weight coefficient corresponding to the multiple target samples as the average distillation loss function corresponding to the training samples; and determine the weighted sum of the average truth loss function and the average distillation loss function as the total loss function.

[0104] The device provided in this embodiment can be used to execute the model training method provided in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0105] Corresponding to the image processing method of the above embodiment, Figure 5 This is a structural block diagram of an image processing device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 5 , the device includes: an acquisition module 501 and a prediction module 502.

[0106] An acquisition module 501 is used to acquire an image to be processed;

[0107] The prediction module 502 is used to input the image to be processed into a deep learning model to obtain a prediction result corresponding to the image to be processed; the deep learning model is obtained by training according to the model training method described in the above method embodiment.

[0108] The device provided in this embodiment can be used to execute the image processing method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0109] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0110] refer to Figure 6, which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0111] like Figure 6 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0112] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0113] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0114] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0115] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0116] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0117] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0120] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0123] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0124] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A model training method, characterized in that: include: Obtain training samples, input the training samples into the student model and the teacher model respectively, and obtain a first output value output by the student model and a second output value output by the teacher model; Determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value, and the true value label of the training sample; A total loss function is determined according to the distillation loss function and a weight coefficient of the distillation loss function, and the student model is trained according to the total loss function.

2. The method according to claim 1, characterized in that The determining the weight coefficient of the distillation loss function according to the first output value, the second output value and the true value label of the training sample includes: Determine a first truth loss function corresponding to the student model and a second truth loss function corresponding to the teacher model according to the first output value, the second output value and the truth label of the training sample; Determine a weight coefficient of the distillation loss function according to the first truth loss function and the second truth loss function.

3. The method according to claim 2, characterized in that The determining, according to the first truth loss function and the second truth loss function, a weight coefficient of the distillation loss function comprises: If the first truth loss function is less than or equal to the second truth loss function, the weight coefficient of the distillation loss function is determined to be a first preset value; the first preset value is less than 1 and greater than or equal to 0.

4. The method according to claim 2 or 3, characterized in that: The determining, according to the first truth loss function and the second truth loss function, a weight coefficient of the distillation loss function comprises: If the first truth loss function is greater than the second truth loss function, the cosine similarity between the first vector and the second vector is determined, and the weight coefficient of the distillation loss function is determined according to the cosine similarity; the first vector is determined according to the first output value and the truth label, and the second vector is determined according to the second output value and the truth label.

5. The method according to claim 4, characterized in that Determining the weight coefficient of the distillation loss function according to the cosine similarity includes: If the cosine similarity is less than or equal to 0, the sum of the cosine similarity and a second preset value is determined as the weight coefficient of the distillation loss function; the second preset value is greater than or equal to 1.

6. The method according to claim 4, characterized in that Determining the weight coefficient of the distillation loss function according to the cosine similarity includes: If the cosine similarity is greater than 0, the weight coefficient of the distillation loss function is determined to be a third preset value; the third preset value is greater than or equal to 1 and less than 2.

7. The method according to claim 2 or 3, characterized in that: The determining of the total loss function according to the distillation loss function and the weight coefficient of the distillation loss function includes: Determining a product of the distillation loss function and a weight coefficient of the distillation loss function; Perform a weighted sum of the product and the first true value loss function to obtain a total loss function.

8. The method according to claim 1, characterized in that The training sample includes a plurality of target samples; the first output value includes first target output values ​​corresponding to the plurality of target samples respectively, and the second output value includes second target output values ​​corresponding to the plurality of target samples respectively; The determining the weight coefficient of the distillation loss function according to the first output value, the second output value and the true value label of the training sample includes: For each target sample, determine a first target truth loss function corresponding to the student model and a second target truth loss function corresponding to the teacher model according to a first target output value and a second target output value and a target truth label corresponding to the target sample, and determine a target weight coefficient of a target distillation loss function corresponding to the target sample according to the first target truth loss function and the second target truth loss function; The determining of the total loss function according to the distillation loss function and the weight coefficient of the distillation loss function includes: Determine the average of the first target true value loss functions corresponding to the plurality of target samples as the average true value loss function corresponding to the training sample; Determine the average value of the product of the target distillation loss function and the target weight coefficient respectively corresponding to the plurality of target samples as the average distillation loss function corresponding to the training sample; A weighted sum of the average true value loss function and the average distillation loss function is determined as the total loss function.

9. An image processing method, characterized in that: include: Get the image to be processed; Inputting the image to be processed into a deep learning model to obtain a prediction result corresponding to the image to be processed; The deep learning model is obtained by training according to the model training method according to any one of claims 1 to 8.

10. A model training device, characterized in that: include: An input module is used to obtain training samples, input the training samples into the student model and the teacher model respectively, and obtain a first output value output by the student model and a second output value output by the teacher model; A determination module, configured to determine a distillation loss function and a weight coefficient of the distillation loss function according to the first output value, the second output value, and the true value label of the training sample; A training module is used to determine a total loss function according to the distillation loss function and a weight coefficient of the distillation loss function, and to train the student model according to the total loss function.

11. An image processing device, characterized in that: include: An acquisition module, used for acquiring an image to be processed; A prediction module is used to input the image to be processed into a deep learning model to obtain a prediction result corresponding to the image to be processed; the deep learning model is trained according to the model training method according to any one of claims 1 to 8.

12. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 9 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images