Student model training method, device, equipment and storage medium

Abstract

The present invention discloses a student model training method, apparatus, device, and storage medium. The method comprises: in one iteration cycle, inputting a sample image into a student network and a corresponding teacher network, respectively; wherein the student network and the teacher network are used to perform target detection on the sample image; determining a first loss value based on the label data of the sample image and the prediction result of the student network, and determining a second loss value based on the label data of the sample image and the prediction result of the teacher network; determining a first distillation effect of knowledge distillation on the student network based on the first loss value and the second loss value; determining a target student loss value corresponding to the student network based on the first distillation effect; and adjusting the network parameters of the student network based on the target student loss value. Embodiments of the present invention improve the detection accuracy of the trained student model.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a student model training method, device, equipment and storage medium. Background Art

[0002] With the continuous development of deep learning, object detection technology has rapidly advanced in recent years, and object detection models have become increasingly complex. While object detection performance continues to improve, the detection process also becomes more time-consuming. To address this issue, a distillation-based approach was proposed. This approach uses a student model (a relatively simple object detection model) to learn the network black-box knowledge of a teacher model (a complex object detection model) through distillation.

[0003] However, in the existing technology, when the student model learns the teacher model through the distillation method, the detection ability of the student model is limited by the detection ability of the teacher model, resulting in the student model being completely dependent on the teacher model, which limits the improvement of the learning model's accuracy during the training process. Summary of the Invention

[0004] The present invention provides a student model training method, apparatus, equipment and storage medium to improve the detection accuracy of the trained student model.

[0005] According to one aspect of the present invention, a student model training method is provided, the method comprising:

[0006] In one iteration cycle, a sample image is input into a student network and a corresponding teacher network respectively; wherein the student network and the teacher network are used to perform target detection on the sample image;

[0007] Determining a first loss value based on the label data of the sample image and the prediction result of the student network, and determining a second loss value based on the label data of the sample image and the prediction result of the teacher network;

[0008] Determining a first distillation effect degree of knowledge distillation on the student network according to the first loss value and the second loss value;

[0009] determining a target student loss value corresponding to the student network based on the first distillation effect degree;

[0010] The network parameters of the student network are adjusted according to the target student loss value.

[0011] According to another aspect of the present invention, there is provided a student model training device, the device comprising:

[0012] A sample image input module is used to input the sample image into the student network and the corresponding teacher network in one iteration cycle; wherein the student network and the teacher network are used to perform target detection on the sample image;

[0013] a loss value determination module, configured to determine a first loss value based on the label data of the sample image and the prediction result of the student network, and to determine a second loss value based on the label data of the sample image and the prediction result of the teacher network;

[0014] A first effect degree determination module is used to determine a first distillation effect degree of knowledge distillation on the student network according to the first loss value and the second loss value;

[0015] a target student loss value determining module, configured to determine a target student loss value corresponding to the student network based on the first distillation effect degree;

[0016] A network parameter adjustment module is used to adjust the network parameters of the student network according to the target student loss value.

[0017] According to another aspect of the present invention, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the student model training method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the student model training method described in any embodiment of the present invention when executed.

[0022] This embodiment determines a first loss value based on the sample image's label data and the student network's prediction results, and a second loss value based on the sample image's label data and the teacher network's prediction results. Based on the first and second loss values, the degree of knowledge distillation on the student network is determined. Based on the first distillation degree, a target student loss value corresponding to the student network is determined. The network parameters of the student network are adjusted based on the target student loss value. By considering the degree of knowledge distillation effect of the teacher network on the student network during training, this approach dynamically adjusts the target student loss value of the student network during training. This ensures that the student network's detection capabilities are not entirely dependent on the teacher network, thereby improving the detection accuracy of the trained student model.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a flow chart of a student model training method provided according to the first embodiment of the present invention;

[0026] Figure 2 This is a flow chart of a student model training method provided according to the second embodiment of the present invention;

[0027] Figure 3 This is a flow chart of a student model training method provided in accordance with the third embodiment of the present invention;

[0028] Figure 4 2 is a schematic structural diagram of a student model training device provided according to a fourth embodiment of the present invention;

[0029] Figure 5 It is a structural diagram of an electronic device for implementing the student model training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] Figure 1 A flowchart of a student model training method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a student model and a teacher model learn together based on knowledge distillation. The method can be executed by a student model training device, which can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] S110. In one iteration cycle, the sample image is input into the student network and the corresponding teacher network respectively; wherein the student network and the teacher network are used to perform target detection on the sample image.

[0035] The student network can be a relatively simple object detection model, while the teacher network can be a more complex one. The specific settings can be pre-configured by relevant technical personnel. For example, the student network can be a ResNet-34, and the corresponding teacher network can be a ResNet-50. The teacher network can be a network model that has been weighted using a pre-trained teacher model; the student network can be an unweighted network model. During the knowledge distillation process, the teacher network guides the student network.

[0036] The sample images may be labeled sample training sets and / or sample test sets. A set number of sample training sets are input to the student network and the teacher network. The number of sample training sets can be pre-set by relevant technical personnel. For example, based on actual needs, 80% of the sample images can be input as sample training sets to the student network and the corresponding teacher network, and 20% of the sample images can be used as sample test sets for subsequent model testing.

[0037] It should be noted that during the model training process, multiple epochs of model training are required. One epoch represents one round of iterative model training, i.e., one round of iterative training on the entire training set. Within each epoch, at least one forward and backward propagation pass on the training set is completed within the current epoch, based on the set batch_size.

[0038] For example, in the current iteration, sample images serving as a sample training set are fed into the student network and the corresponding teacher network. The student and teacher networks simultaneously load the sample images. The student and teacher networks train on the sample images simultaneously.

[0039] S120. Determine a first loss value based on the label data of the sample image and the prediction result of the student network, and determine a second loss value based on the label data of the sample image and the prediction result of the teacher network.

[0040] The label data of the sample images may be pre-labeled by relevant technical personnel before the sample images are input into the student network and / or the teacher network.

[0041] For example, the student network is trained based on sample images with labeled data as input, and prediction results corresponding to the sample images can be obtained; and the teacher network is trained based on sample images with labeled data as input, and prediction results corresponding to the sample images can be obtained. The first loss value can be a loss value obtained based on the prediction results of the student network and the true values ​​in the labeled data of the sample images. The second loss value can be a loss value obtained based on the prediction results of the teacher network and the true values ​​in the labeled data of the sample images.

[0042] In an optional embodiment, the prediction result of the student network includes a first classification prediction value and a first regression prediction value; accordingly, determining the first loss value corresponding to the student network includes: determining the first classification loss value based on the first classification prediction value and the category label value in the label data; and determining the first regression loss value based on the first regression loss value and the position label value in the label data; generating the first loss value based on the first classification loss value and the first regression loss value.

[0043] The first classification prediction value may be a predicted classification result obtained by training a student network on a sample image. For example, the first classification prediction value may be a probability value predicted by the student network. The category label value may be the true value of the category to which the sample image belongs. The first regression prediction value may be a predicted coordinate result obtained by training a student network on a sample image. The coordinates may be the predicted position coordinates of the detection frame corresponding to the sample image. The position label value may be the true coordinate position corresponding to the detection frame of the sample image.

[0044] The first classification loss value can be determined based on the first classification prediction value and the category label value in the label data, based on a preset classification loss function. The classification loss function can be preset by relevant technical personnel. For example, the classification loss function can be an exponential loss function and a logarithmic likelihood loss function, etc., which is not limited in this embodiment. Exemplarily, the first classification prediction value and the category label value can be input into a preset classification loss function, and the classification loss function is used to calculate and obtain the first classification loss value.

[0045] The first regression loss value can be determined based on the first regression prediction value and the position label value in the label data, based on a preset regression loss function. The regression loss function can be preset by relevant technical personnel. For example, the regression loss function can be a squared loss function, an absolute loss function, a Huber loss function, etc., which is not limited in this embodiment. Exemplarily, the first regression prediction value and the position label value can be input into a preset regression loss function, and the first regression loss value is calculated and obtained by the regression loss function.

[0046] A first loss value is generated based on the first classification loss value and the first regression loss value. For example, the first classification loss value and the first regression loss value may be added together, and the result of the addition of the two loss values ​​may be used as the first loss value. Alternatively, a weighted summation operation may be performed on the first classification loss value and the first regression loss value based on a preset weight threshold, and the loss value obtained by weighted summation of the two loss values ​​based on the preset weight threshold may be used as the first loss value.

[0047] This optional embodiment generates a first loss value based on the first classification loss value and the first regression loss value, thereby accurately determining the first loss value in the student network training process, thereby laying the foundation for further accurate determination of the target student loss value in the subsequent process.

[0048] In an optional embodiment, the prediction result of the teacher network includes a second classification prediction value and a second regression prediction value; accordingly, determining the second loss value corresponding to the teacher network includes: determining the second classification loss value based on the second classification prediction value and the category label value in the label data; and determining the second regression loss value based on the second regression prediction value and the position label value in the label data; generating a second loss value based on the second classification loss value and the second regression loss value.

[0049] The second classification prediction value may be a predicted classification result obtained by the teacher network for the sample image. For example, the second classification prediction value may be a probability value predicted by the teacher network. The category label value may be the true value of the category to which the sample image belongs. The second regression prediction value may be a predicted coordinate result obtained by the teacher network for training the sample image. The coordinates may be the predicted position coordinates of the detection frame corresponding to the sample image. The position label value may be the true coordinate position corresponding to the detection frame of the sample image.

[0050] The second classification loss value can be determined based on the second classification prediction value and the class label value in the label data, based on a preset classification loss function. The classification loss function can be preset by relevant technical personnel. For example, the classification loss function can be an exponential loss function and a logarithmic likelihood loss function, etc., which is not limited in this embodiment. Exemplarily, the second classification prediction value and the class label value can be input into a preset classification loss function, and the second classification loss value is calculated and obtained by the classification loss function.

[0051] The second regression loss value can be determined based on the second regression prediction value and the position label value in the label data, based on a preset regression loss function. The regression loss function can be preset by relevant technical personnel. For example, the regression loss function can be a squared loss function, an absolute loss function, a Huber loss function, etc., which is not limited in this embodiment. Exemplarily, the second regression prediction value and the position label value can be input into a preset regression loss function, and the second regression loss value is calculated and obtained by the regression loss function.

[0052] A second loss value is generated based on the second classification loss value and the second regression loss value. For example, the second classification loss value and the second regression loss value may be added together, and the result of the addition of the two loss values ​​may be used as the second loss value. Alternatively, a weighted summation operation may be performed on the second classification loss value and the second regression loss value based on a preset weight threshold, and the loss value obtained by weighted summation of the two loss values ​​based on the preset weight threshold may be used as the second loss value.

[0053] This optional embodiment generates a second loss value based on the second classification loss value and the second regression loss value, thereby achieving accurate determination of the second loss value in the teacher network training process, thereby laying the foundation for further accurate determination of the target student loss value in the subsequent process.

[0054] S130. Determine the first distillation effect of knowledge distillation on the student network based on the first loss value and the second loss value.

[0055] The first degree of distillation can be the degree of knowledge distillation from the teacher network to the student network. It should be noted that as the student network learns from the teacher network through distillation, its object detection capability gradually improves. To prevent the student network's object detection capability from being limited by the teacher network's, the influence of the teacher network on the student network's knowledge distillation can be dynamically adjusted during the knowledge distillation process based on the degree of distillation.

[0056] For example, if the first loss value is not less than the second loss value, it can be considered that the teacher network has good target detection capabilities during the prediction process. Therefore, the first distillation effect can be dynamically adjusted to increase the teacher network's contribution to the student network's knowledge distillation process. If the first loss value is less than the second loss value, it can be considered that the student network has good target detection capabilities during training. Therefore, the first distillation effect can be dynamically adjusted to reduce the teacher network's contribution to the student network's knowledge distillation process. The adjustment method and scale of the first distillation effect can be pre-set by relevant technical personnel.

[0057] S140. Determine a target student loss value corresponding to the student network based on the first distillation effect degree.

[0058] The target student loss value may be a loss value used to determine whether the student network has converged during the iteration process of the student network.

[0059] For example, the distillation loss value can be adjusted by adjusting the first distillation effect level to obtain a target student loss value. The distillation loss value can be calculated based on parameters such as feature values ​​generated during the training of the student network and the teacher network using a preset distillation loss function. The distillation loss function can be pre-set by relevant technical personnel, and specifically, the distillation loss value can be calculated using an existing distillation loss function, which is not limited in this embodiment. Optionally, the target student loss value can be the product of the first distillation effect level and the distillation loss value.

[0060] S150. Adjust the network parameters of the student network according to the target student loss value.

[0061] The network parameters may include network weight parameters and network feature parameters. For example, the target student loss value may be used to determine whether the student network has converged. If so, the converged student network is used as the student model for target detection. If not, the target student loss value is used to adjust the network parameters by backpropagating the student network based on a preset backpropagation algorithm.

[0062] It should be noted that the convergence condition of the student network can be that the target student loss value tends to be stable or that the difference from the target student loss value obtained in the previous iteration tends to be stable, or that the iterative training of the student network is stopped after reaching the set iteration cycle threshold. This implementation does not impose any restrictions on this.

[0063] This embodiment determines a first loss value based on the sample image's label data and the student network's prediction results, and a second loss value based on the sample image's label data and the teacher network's prediction results. Based on the first and second loss values, the degree of knowledge distillation on the student network is determined. Based on the first distillation degree, a target student loss value corresponding to the student network is determined. The network parameters of the student network are adjusted based on the target student loss value. By considering the degree of knowledge distillation effect of the teacher network on the student network during training, this approach dynamically adjusts the target student loss value of the student network during training. This ensures that the student network's detection capabilities are not entirely dependent on the teacher network, thereby improving the detection accuracy of the trained student model.

[0064] Example 2

[0065] Figure 2 This is a flow chart of a student model training method provided in Example 2 of the present invention. This embodiment is optimized and improved based on the above technical solutions.

[0066] Furthermore, the step of "determining the target student loss value corresponding to the student network based on the first distillation effect degree" is refined into "determining the first distillation weight value corresponding to the student network according to the first distillation effect degree; determining the first distillation loss value generated by the student network training; determining the target student loss value corresponding to the student network according to the first loss value, the first distillation loss value and the first distillation weight value." to improve the method of determining the target student loss value.

[0067] like Figure 2 As shown, the method includes the following specific steps:

[0068] S210. In one iteration cycle, the sample image is input into the student network and the corresponding teacher network respectively; wherein the student network and the teacher network are used to perform target detection on the sample image.

[0069] S220. Determine a first loss value based on the label data of the sample image and the prediction result of the student network, and determine a second loss value based on the label data of the sample image and the prediction result of the teacher network.

[0070] S230. Determine the first distillation effect of knowledge distillation on the student network based on the first loss value and the second loss value.

[0071] S240: Determine a first distillation weight value corresponding to the student network according to the first distillation effect degree.

[0072] The first distillation weight value corresponding to the student network can be determined as follows:

[0073]

[0074] Among them, w stu_dil Can represent the first distillation weight value, loss stu Can represent the first loss value, loss tch Can represent the second loss value. loss stu and loss tch The comparison between the first and second loss values ​​indicates the extent to which the teacher network contributes to the student network's knowledge distillation. If the first loss value is not less than the second loss value, the distillation effect on the student network is greater; if the first loss value is less than the second loss value, the distillation effect on the student network is less. Specifically, the first distillation weight value can be determined based on the extent of the first distillation effect of the teacher network on the student network.

[0075] S250: Determine a first distillation loss value generated by student network training.

[0076] Exemplarily, the first distillation loss value can be calculated based on a preset distillation loss function according to training parameters, such as eigenvalues, generated during the training process of the student network and the teacher network.

[0077] In an optional embodiment, determining a first distillation loss value generated by student network training includes: determining a first predicted feature value obtained after feature extraction of a network layer of the student network, and determining a second predicted feature value obtained after feature extraction of a network layer of the teacher network; and determining a first distillation loss value based on the first predicted feature value and the second predicted feature value.

[0078] The network layers of the student network and the teacher network may include a feature layer and a bottleneck layer. The first predicted feature value may be obtained by simultaneously inferring or predicting the sample image using the teacher network and the student network, and obtaining feature values ​​corresponding to the feature layer and bottleneck layer of the student network, as well as feature values ​​of the detection box corresponding to the predicted sample image. The second predicted feature value may be obtained by simultaneously inferring or predicting the sample image using the teacher network and the student network, and obtaining feature values ​​corresponding to the feature layer and bottleneck layer of the teacher network, as well as feature values ​​of the detection box corresponding to the predicted sample image.

[0079] Exemplarily, an error calculation may be performed based on a preset distillation loss function according to the first predicted eigenvalue and the second predicted eigenvalue, and the error calculation result of the distillation loss function may be used as the first distillation loss value.

[0080] S260: Determine a target student loss value corresponding to the student network according to the first loss value, the first distillation loss value, and the first distillation weight value.

[0081] The first loss value may be the sum of the first classification loss value and the first regression loss value. The target student loss value may be the sum of the first loss value and the first distillation loss value adjusted based on the first distillation weight value.

[0082] In an optional embodiment, a target loss value corresponding to the student network is determined based on the first loss value, the first distillation loss value and the first distillation weight value, including: adjusting the first distillation loss value based on the first distillation weight value; and obtaining the target student loss value corresponding to the student network based on the sum of the first loss value and the adjusted first distillation loss value.

[0083] The target student loss value can be determined in the following way:

[0084] loss stu_sum =loss stu_class +loss stu_bbox +w stu_dil *loss 蒸馏损失

[0085] Among them, loss stu_sum Can represent the target student loss value, loss stu_class Can represent the first classification loss value, loss stu_bbox Can represent the first regression loss value, w stu_dil Can represent a distillation weight value, loss 蒸馏损失 The first distillation loss value may be represented.

[0086] For example, the first distillation loss value can be adjusted using the first distillation weight value, and the adjusted first distillation loss value is added to the first loss value to obtain the target student loss value. The first loss value is the sum of the first regression loss value and the first classification loss value.

[0087] This optional embodiment adjusts the first distillation loss value based on the first distillation weight value; obtains the target student loss value corresponding to the student network based on the sum of the first loss value and the adjusted first distillation loss value, thereby achieving dynamic determination of the target student loss value and being able to dynamically adjust it according to the degree of distillation effect of the teacher network on the student network.

[0088] S270. Adjust the network parameters of the student network according to the target student loss value.

[0089] This embodiment determines a first distillation weight value corresponding to the student network based on the first distillation effect; determines a first distillation loss value generated by student network training; and determines a target student loss value corresponding to the student network based on the first loss value, the first distillation loss value, and the first distillation weight value. This approach dynamically adjusts the student network's learning ability based on the degree of knowledge distillation performed by the teacher network on the student network, avoiding a situation where the student network's learning ability is completely limited by the teacher network's detection ability when learning the teacher network's knowledge. Furthermore, by dynamically adjusting the degree of distillation, the student network's ability to learn from the teacher network can be further adjusted, making the student network's detection ability less dependent on the teacher network and improving the detection accuracy of the trained student model.

[0090] Example 3

[0091] Figure 3 This is a flow chart of a student model training method provided in Example 3 of the present invention. This embodiment is optimized and improved based on the above technical solutions.

[0092] Furthermore, the method also includes: generating a target teacher loss value based on the second loss value; adjusting the network parameters of the teacher network based on the target teacher loss value to update the teacher network in this iteration cycle.

[0093] like Figure 3 As shown, the method includes the following specific steps:

[0094] S310. In one iteration cycle, the sample image is input into the student network and the corresponding teacher network respectively; wherein the student network and the teacher network are used to perform target detection on the sample image.

[0095] S320. Determine a first loss value based on the label data of the sample image and the prediction result of the student network, and determine a second loss value based on the label data of the sample image and the prediction result of the teacher network.

[0096] S330. Determine the first distillation effect of knowledge distillation on the student network based on the first loss value and the second loss value.

[0097] S340: Determine a target student loss value corresponding to the student network based on the first distillation effect degree.

[0098] S350. Adjust the network parameters of the student network according to the target student loss value.

[0099] S360. Generate a target teacher loss value based on the second loss value.

[0100] It's important to note that in existing knowledge distillation solutions, distillation guidance only applies to the student network and not the teacher network. Distillation also doesn't dynamically adjust the teacher network based on the target detection capabilities of the teacher and student networks. The teacher network doesn't learn during the distillation process, so the knowledge distillation process doesn't further improve the target detection capabilities of the teacher network.

[0101] It is understandable that if we want to solve the above problems, we can dynamically adjust the teacher network during the knowledge distillation process so that the target detection ability of the teacher network can be further improved.

[0102] In an optional embodiment, if the historical accuracy corresponding to the student network in the previous iteration cycle is less than a preset accuracy threshold, a target teacher loss value is generated based on the second loss value, including: using the second loss value as the target teacher loss value.

[0103] The previous iteration cycle may be a cycle of an iterative training round before the current iteration cycle, and the historical accuracy rate may be the accuracy rate of target detection of the student network obtained by training in the previous iteration cycle.

[0104] The historical accuracy of the student network can be determined by inputting a pre-divided sample test set into the student network trained in the previous iteration. The historical accuracy of the student network in the previous iteration is determined based on the test results of the student network on the sample test set.

[0105] The accuracy threshold can be pre-set by relevant technical personnel. For example, the accuracy threshold can be set to 0.78.

[0106] For example, if the historical accuracy of the student network in the previous iteration cycle is less than a preset accuracy threshold, the second loss value is used as the target teacher loss value. The target teacher loss value can be a loss value used to determine whether the teacher network has converged during the teacher network iteration process.

[0107] In an optional embodiment, if the historical accuracy corresponding to the student network in the previous iteration cycle is not less than a preset accuracy threshold, a target teacher loss value is generated based on the second loss value, including: determining the second distillation effect of knowledge distillation on the teacher network based on the first loss value and the second loss value; determining the second distillation weight value corresponding to the teacher network based on the second distillation effect; determining the second distillation loss value generated by the teacher network training; and determining the target teacher loss value corresponding to the teacher network based on the second loss value, the second distillation loss value and the second distillation weight value.

[0108] The second distillation weight value corresponding to the teacher network can be determined as follows:

[0109]

[0110] Among them, w tch_dil It can represent the second distillation weight value, that is, the weight value of the distillation effect on the teacher network. loss stu Can represent the first loss value, loss tch Can represent the second loss value. loss stu and loss tch The comparison between the two can indicate the extent to which the teacher network contributes to the student network's knowledge distillation. If the second loss value is not less than the first loss value, the distillation effect on the student network is greater; if the second loss value is less than the first loss value, the distillation effect on the student network is less. Specifically, the corresponding second distillation weight value can be determined based on the extent of the second distillation effect on the teacher network.

[0111] Among them, the target teacher loss value can be determined in the following way:

[0112] loss tch_sum =loss tch_class +loss tch_bbox +w tch_dil *loss 蒸馏损失

[0113] Among them, loss tch_sum It can represent the target teacher loss value, that is, the loss value of the distillation effect on the teacher network. tch_class Can represent the second classification loss value, loss tch_bbox Can represent the second regression loss value, w tch_dilCan represent the second distillation weight value, loss 蒸馏损失 The second distillation loss value may be determined in the same manner as the first distillation loss value, which will not be described in detail in this embodiment.

[0114] S370. Adjust the network parameters of the teacher network according to the target teacher loss value to update the teacher network in this iteration cycle.

[0115] The network parameters may include network weight parameters and network feature parameters. For example, the target teacher loss value may be used to determine whether the teacher network has converged; if so, the converged teacher network is used as the teacher model for target detection; if not, the target teacher loss value is used to adjust the network parameters by backpropagating the teacher network based on a preset backpropagation algorithm.

[0116] It should be noted that the convergence condition of the teacher network can be that the target teacher loss value tends to be stable or that the difference between the target teacher loss value obtained in the previous iteration tends to be stable, or that the iterative training of the teacher network is stopped after reaching the set iteration cycle threshold. This implementation does not impose any restrictions on this.

[0117] This embodiment determines the target teacher loss value based on the historical accuracy of the student network in the previous iteration. Based on the target teacher loss value, the teacher network parameters are adjusted to update the teacher network in the current iteration. This allows the teacher network to learn and improve its detection capabilities during the knowledge distillation process, while also alleviating the upper limit of the student network's detection capabilities. Furthermore, during the iteration of the teacher network, the detection capabilities can be dynamically adjusted, allowing the student and teacher networks to learn and improve their detection capabilities together.

[0118] Example 4

[0119] Figure 4 This is a structural diagram of a student model training device provided by the fourth embodiment of the present invention. The student model training device provided by the embodiment of the present invention is applicable to the situation where the student model and the teacher model learn together based on knowledge distillation. The device can be implemented in software and / or hardware. Figure 4 As shown, the device specifically includes: a sample image input module 401, a loss value determination module 402, a first effect degree determination module 403, a target student loss value determination module 404 and a first network parameter adjustment module 405.

[0120] The sample image input module 401 is used to input the sample image into the student network and the corresponding teacher network in one iteration cycle; wherein the student network and the teacher network are used to perform target detection on the sample image;

[0121] a loss value determining module 402, configured to determine a first loss value based on the label data of the sample image and the prediction result of the student network, and to determine a second loss value based on the label data of the sample image and the prediction result of the teacher network;

[0122] A first effect degree determining module 403 is configured to determine a first distillation effect degree of knowledge distillation on the student network based on the first loss value and the second loss value;

[0123] a target student loss value determining module 404, configured to determine a target student loss value corresponding to the student network based on the first distillation effect level;

[0124] The first network parameter adjustment module 405 is configured to adjust the network parameters of the student network according to the target student loss value.

[0125] This embodiment determines a first loss value based on the sample image's label data and the student network's prediction results, and a second loss value based on the sample image's label data and the teacher network's prediction results. Based on the first and second loss values, the degree of knowledge distillation on the student network is determined. Based on the first distillation degree, a target student loss value corresponding to the student network is determined. The network parameters of the student network are adjusted based on the target student loss value. By considering the degree of knowledge distillation effect of the teacher network on the student network during training, this approach dynamically adjusts the target student loss value of the student network during training. This ensures that the student network's detection capabilities are not entirely dependent on the teacher network, thereby improving the detection accuracy of the trained student model.

[0126] Optionally, the target student loss value determination module 404 includes:

[0127] a first weight value determining unit, configured to determine a first distillation weight value corresponding to the student network according to the first distillation effect degree;

[0128] A first distillation loss value determining unit, configured to determine a first distillation loss value generated by the student network training;

[0129] A target student loss value determining unit is configured to determine a target student loss value corresponding to the student network according to the first loss value, the first distillation loss value, and the first distillation weight value.

[0130] Optionally, the target student loss value determination unit includes:

[0131] a first distillation loss value adjusting subunit, configured to adjust the first distillation loss value based on the first distillation weight value;

[0132] The target student loss value determining subunit is used to obtain the target student loss value corresponding to the student network according to the sum of the first loss value and the adjusted first distillation loss value.

[0133] Optionally, the prediction result of the student network includes a first classification prediction value and a first regression prediction value;

[0134] Accordingly, the loss value determination module 402 includes:

[0135] a first classification loss value determining unit, configured to determine the first classification loss value according to the first classification prediction value and the class label value in the label data; and

[0136] a first regression loss value determining unit, configured to determine the first regression loss value according to the first regression prediction value and the position label value in the label data;

[0137] A first loss value determining unit is configured to generate the first loss value according to the first classification loss value and the first regression loss value.

[0138] Optionally, the prediction result of the teacher network includes a second classification prediction value and a second regression prediction value;

[0139] Accordingly, the loss value determination module 402 includes:

[0140] a second classification loss value determining unit, configured to determine the second classification loss value according to the second classification prediction value and the class label value in the label data; and

[0141] a second regression loss value determining unit, configured to determine the second regression loss value according to the second regression prediction value and the position label value in the label data;

[0142] A second loss value determining unit is used to generate the second loss value according to the second classification loss value and the second regression loss value.

[0143] Optionally, the first distillation loss value determining unit includes:

[0144] a predicted characteristic value determination subunit, configured to determine a first predicted characteristic value obtained after feature extraction of the network layer of the student network, and to determine a second predicted characteristic value obtained after feature extraction of the network layer of the teacher network;

[0145] The first distillation loss value determining subunit is configured to determine the first distillation loss value according to the first predicted characteristic value and the second predicted characteristic value.

[0146] Optionally, the device further includes:

[0147] a target teacher loss value generating module, configured to generate a target teacher loss value according to the second loss value;

[0148] The second network parameter adjustment module is used to adjust the network parameters of the teacher network according to the target teacher loss value to update the teacher network in this iteration cycle.

[0149] Optionally, if the historical accuracy of the student network in the previous iteration cycle is less than a preset accuracy threshold, the target teacher loss value generation module includes:

[0150] The first target teacher loss value determination unit is used to use the second loss value as the target teacher loss value.

[0151] Optionally, if the historical accuracy of the student network in the previous iteration cycle is not less than a preset accuracy threshold, the target teacher loss value generation module includes:

[0152] a second effect degree determining unit, configured to determine a second distillation effect degree of knowledge distillation on the teacher network according to the first loss value and the second loss value;

[0153] a second weight value determining unit, configured to determine a second distillation weight value corresponding to the teacher network according to the second distillation effect degree;

[0154] a second distillation loss value determining unit, configured to determine a second distillation loss value generated by the teacher network training;

[0155] A second target teacher loss value determining unit is used to determine a target teacher loss value corresponding to the teacher network according to the second loss value, the second distillation loss value and the second distillation weight value.

[0156] The student model training device provided in the embodiment of the present invention can execute the student model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0157] Example 5

[0158] Figure 5A schematic diagram of the structure of an electronic device 50 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0159] like Figure 5 As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52, a random access memory (RAM) 53, etc., which is communicatively connected to the at least one processor 51. The memory stores a computer program that can be executed by the at least one processor. The processor 51 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 52 or the computer program loaded from the storage unit 58 into the random access memory (RAM) 53. Various programs and data required for the operation of the electronic device 50 can also be stored in the RAM 53. The processor 51, ROM 52, and RAM 53 are connected to each other via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0160] Multiple components in the electronic device 50 are connected to the I / O interface 55, including an input unit 56, such as a keyboard, a mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a magnetic disk, an optical disk, etc.; and a communication unit 59, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0161] The processor 51 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 51 performs the various methods and processes described above, such as the student model training method.

[0162] In some embodiments, the student model training method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 58. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 50 via the ROM 52 and / or the communication unit 59. When the computer program is loaded into the RAM 53 and executed by the processor 51, one or more steps of the student model training method described above can be performed. Alternatively, in other embodiments, the processor 51 can be configured to perform the student model training method in any other appropriate manner (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0168] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0169] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0170] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A student model training method, characterized in that: include: In one iteration cycle, a sample image is input into a student network and a corresponding teacher network respectively; wherein the student network and the teacher network are used to perform target detection on the sample image; Determining a first loss value based on the label data of the sample image and the prediction result of the student network, and determining a second loss value based on the label data of the sample image and the prediction result of the teacher network; Determining a first distillation effect degree of knowledge distillation on the student network based on the first loss value and the second loss value; wherein the first distillation effect degree is the effect degree of knowledge distillation of the teacher network on the student network, and is used to characterize the magnitude relationship between the first loss value and the second loss value; the first distillation effect degree is used to dynamically adjust the influence of the teacher network on the knowledge distillation of the student network; determining a target student loss value corresponding to the student network based on the first distillation effect degree; Adjusting the network parameters of the student network according to the target student loss value; The step of determining a target student loss value corresponding to the student network based on the first distillation effect level includes: Determining a first distillation weight value corresponding to the student network according to the first distillation effect degree; Determining a first distillation loss value resulting from the student network training; A target student loss value corresponding to the student network is determined according to the first loss value, the first distillation loss value, and the first distillation weight value.

2. The method according to claim 1, characterized in that Determining a target student loss value corresponding to the student network according to the first loss value, the first distillation loss value, and the first distillation weight value includes: adjusting the first distillation loss value based on the first distillation weight value; A target student loss value corresponding to the student network is obtained according to the sum of the first loss value and the adjusted first distillation loss value.

3. The method according to claim 1, characterized in that The prediction result of the student network includes a first classification prediction value and a first regression prediction value; Accordingly, determining a first loss value according to the label data of the sample image and the prediction result of the student network includes: Determining a first classification loss value based on the first classification prediction value and the class label value in the label data; and Determining a first regression loss value based on the first regression prediction value and the position label value in the label data; The first loss value is generated according to the first classification loss value and the first regression loss value.

4. The method according to claim 1, wherein The prediction result of the teacher network includes a second classification prediction value and a second regression prediction value; Accordingly, determining a second loss value according to the label data of the sample image and the prediction result of the teacher network includes: Determining a second classification loss value based on the second classification prediction value and the class label value in the label data; and Determining a second regression loss value based on the second regression prediction value and the position label value in the label data; The second loss value is generated according to the second classification loss value and the second regression loss value.

5. The method according to claim 1, characterized in that Determining a first distillation loss value generated by the student network training includes: Determine a first predicted eigenvalue obtained after feature extraction of a network layer of the student network, and determine a second predicted eigenvalue obtained after feature extraction of a network layer of the teacher network; The first distillation loss value is determined according to the first predicted characteristic value and the second predicted characteristic value.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: generating a target teacher loss value according to the second loss value; According to the target teacher loss value, the network parameters of the teacher network are adjusted to update the teacher network in this iteration cycle.

7. The method according to claim 6, characterized in that If the historical accuracy of the student network in the previous iteration cycle is less than a preset accuracy threshold, generating a target teacher loss value based on the second loss value includes: The second loss value is used as the target teacher loss value.

8. The method according to claim 6, characterized in that If the historical accuracy of the student network in the previous iteration cycle is not less than a preset accuracy threshold, generating a target teacher loss value based on the second loss value includes: Determining a degree of second distillation effect of knowledge distillation on the teacher network according to the first loss value and the second loss value; Determining a second distillation weight value corresponding to the teacher network according to the second distillation effect degree; determining a second distillation loss value resulting from the teacher network training; Determine a target teacher loss value corresponding to the teacher network according to the second loss value, the second distillation loss value, and the second distillation weight value.

9. A student model training device, characterized in that: include: A sample image input module is used to input the sample image into the student network and the corresponding teacher network in one iteration cycle; wherein the student network and the teacher network are used to perform target detection on the sample image; a loss value determination module, configured to determine a first loss value based on the label data of the sample image and the prediction result of the student network, and to determine a second loss value based on the label data of the sample image and the prediction result of the teacher network; A first effect degree determination module is configured to determine a first distillation effect degree of knowledge distillation on the student network based on the first loss value and the second loss value; wherein the first distillation effect degree is the effect degree of the teacher network on the knowledge distillation of the student network, and is used to characterize the magnitude relationship between the first loss value and the second loss value; the first distillation effect degree is used to dynamically adjust the influence of the teacher network on the knowledge distillation of the student network; a target student loss value determining module, configured to determine a target student loss value corresponding to the student network based on the first distillation effect degree; A first network parameter adjustment module, configured to adjust the network parameters of the student network according to the target student loss value; The target student loss value determination module includes: a first weight value determining unit, configured to determine a first distillation weight value corresponding to the student network according to the first distillation effect degree; A first distillation loss value determining unit, configured to determine a first distillation loss value generated by the student network training; A target student loss value determining unit is configured to determine a target student loss value corresponding to the student network according to the first loss value, the first distillation loss value, and the first distillation weight value.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the student model training method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the student model training method according to any one of claims 1 to 8 when executed.

Images