Face detection method in cockpit, target detection model training method and device

Abstract

The specification discloses a face detection method in a cockpit, a target detection model training method and device, and belongs to the technical field of computers, wherein the target detection model training method based on knowledge distillation comprises the following steps: determining the confidence of a prediction box output by a teacher network and a student network for a same target object in a current batch of sample images; determining the loss weight of a regression branch corresponding to the target object according to the confidence of the prediction box output by the teacher network and the student network for the same target object; and adjusting the loss value of a regression loss function according to the loss weight of the regression branch corresponding to each target object in the current batch of sample images, so as to train the student network based on knowledge distillation.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and particularly relates to a face detection method in a cockpit, and a target detection model training method and device. BACKGROUND

[0002] Deep neural networks have achieved great success in the AI field of computer vision, natural language processing, speech recognition, etc., but these deep neural networks require huge computing power, which seriously hinders the use of deep neural networks in resource-constrained scenarios such as intelligent cockpit of a car. However, with the rapid development of the intelligent automobile field, more and more car companies are eager to inject AI capabilities into the intelligent cockpit scenario. In order to meet the running of deep neural network models on a car machine with limited computing resources and ensure the real-time performance and accuracy, model compression has become a crucial technology. Common model compression techniques include model quantization, model pruning, and knowledge distillation, etc. The knowledge distillation (KD) technology is to transfer the knowledge learned by a complex model to a simple model, thereby improving the accuracy of the simple model.

[0003] The prior art compares the distance from the prediction frame output by the teacher network and the student network to the real frame to determine the prediction of the target frame by the teacher network and the student network. The teacher network is better than the student network, but it does not mean that the prediction of the teacher network for each sample is better than that of the student network. However, for a certain sample, when the prediction result of the teacher network is not as good as that of the student network, knowledge distillation may cause the student model to overfit. Since the existing target detection method based on knowledge distillation does not consider the prediction of the teacher network (complex network) and the student network (simple network) for each sample, the effect of knowledge distillation is reduced, resulting in low accuracy of the student network learned through knowledge distillation. SUMMARY

[0004] In view of the above technical problems of the prior art, the embodiments of the present specification provide a face detection method in a cockpit, and a target detection model training method and device.

[0005] In a first aspect, the embodiments of the present specification provide a target detection model training method based on knowledge distillation, comprising:

[0006] determining the confidence of the prediction frame output by the teacher network and the student network for the same target object in the current batch of sample images;

[0007] determining the loss weight of the regression branch corresponding to the target object according to the confidence of the prediction frame output by the teacher network and the student network for the same target object;

[0008] According to a loss weight of each target object in the current batch of sample images corresponding to the regression branch, the loss value of the regression loss function is adjusted, and the student network is trained based on knowledge distillation.

[0009] Optionally, the loss weight of each target object in the current batch of sample images corresponding to the regression branch is determined according to the confidence of the prediction box output by the teacher network and the student network for the same target object.

[0010] Each target object in the current batch of sample images is taken as a target object, and the current batch of sample images includes a plurality of sample images, and at least one target object exists in each sample image.

[0011] The first confidence score of the prediction box output by the teacher network for the target object is obtained by performing forward calculation on the teacher network.

[0012] The second confidence score of the prediction box output by the student network for the target object is obtained by performing forward calculation on the student network.

[0013] The loss weight of the target object corresponding to the regression branch is obtained according to the first confidence score and the second confidence score.

[0014] Optionally, the loss weight of the target object corresponding to the regression branch is obtained according to the first confidence score and the second confidence score, including:

[0015] The confidence ratio of the first confidence score and the second confidence score is determined.

[0016] According to the size relationship between the confidence ratio and a reference weight threshold, the loss weight of the target object corresponding to the regression branch is constrained within a preset weight range.

[0017] Optionally, the reference weight threshold is set to 3-7.

[0018] Optionally, the loss weight of each target object in the current batch of sample images corresponding to the regression branch is adjusted to adjust the loss value of the regression loss function, including:

[0019] For each target object in the current batch of sample images, the original loss value of the target object corresponding to the regression branch is weighted calculated according to the loss weight of the target object corresponding to the regression branch, to obtain an adjusted loss value of the target object corresponding to the regression branch.

[0020] The adjusted loss value of each target object in the current batch of sample images corresponding to the regression branch is added and calculated, and the current loss value of the regression loss function is determined according to the addition calculation result.

[0021] Optionally, after determining the current loss value of the regression loss function according to the addition calculation result, further comprising:

[0022] determining whether the regression loss function meets a convergence condition according to the current loss value;

[0023] if the convergence condition is met, ending the training of the student network based on knowledge distillation to obtain a target detection model after knowledge distillation, otherwise, continuing to train the student network based on knowledge distillation using the next batch of sample images.

[0024] Optionally, further comprising: constructing a complex network model and a simple network model in advance; pre-training the complex network model to generate the teacher network; and taking the simple network model as the student network, wherein the prediction accuracy of the teacher network is higher than that of the student network.

[0025] In a second aspect, the embodiments of the present specification provide a face detection method in a cockpit, applied to a vehicle, and the method comprises:

[0026] detecting a face in the cockpit of the vehicle by using a face detection model, wherein the face detection model is obtained by training the target detection model training method based on knowledge distillation according to any one of the first aspect.

[0027] performing one or more of the following tasks for the driver according to the face detection result: a face recognition task, a face key point detection task, a live detection task, and a fatigue monitoring task.

[0028] In a third aspect, the embodiments of the present specification provide a target detection model training device based on knowledge distillation, comprising:

[0029] a confidence determination unit configured to determine the confidence of the prediction frame output by the teacher network and the student network for the same target object in the current batch of sample images;

[0030] a weight adjustment unit configured to determine the loss weight of the regression branch corresponding to the target object according to the confidence of the prediction frame output by the teacher network and the student network for the same target object;

[0031] a loss calculation unit configured to adjust the loss value of the regression loss function according to the loss weight of the regression branch corresponding to each target object in the current batch of sample images, the regression loss function being used to train the student network based on knowledge distillation.

[0032] In a fourth aspect, the embodiments of the present specification provide a face detection device in a cockpit, applied to a vehicle, and the device comprises:

[0033] The face detection unit is configured to perform face detection in a driver cabin of a vehicle by using a face detection model, wherein the face detection model is trained by the knowledge distillation based target detection model training method in any one of the first aspect.

[0034] The task execution unit is configured to perform one or more of the following tasks for the driver according to the face detection result: a face recognition task, a face key point detection task, a living body detection task, and a fatigue monitoring task.

[0035] In a fifth aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method in any one of the first aspect or the steps of the method in any one of the second aspect when executing the program.

[0036] In a sixth aspect, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the steps of the method in any one of the first aspect or the steps of the method in any one of the second aspect.

[0037] The one or more technical solutions provided by the embodiments of the present specification at least achieve the following technical effects or advantages:

[0038] According to the confidence of the prediction frame output by the teacher network and the student network for the same target object, the loss weight of the target object corresponding to the regression branch is determined; and according to the loss weight of each target object corresponding to the regression branch in the current batch of sample images, the loss value of the regression loss function used for training the student network based on knowledge distillation is adjusted, so that the process of training the student network based on knowledge distillation is realized, and the prediction of each object in each sample image by the teacher network and the student network is considered, and the loss weight of each object corresponding to the regression branch is dynamically adjusted. The loss weight participates in the calculation of the regression loss function, so that the calculation result of the regression loss function fully reflects the good or bad of the prediction of each object in each sample image by the teacher network and the student network. Therefore, the prediction accuracy of the target detection model trained based on knowledge distillation can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments with reference made to the accompanying drawings. The drawings are for purposes of illustration only and are not intended to be limiting in any respect. Like reference numerals are used throughout the drawings to indicate similar or identical parts. In the drawings:

[0040] Figure 1A flowchart of a target detection model training method based on knowledge distillation provided by an embodiment of the present specification is shown in the following figure:

[0041] Figure 2 A flowchart of a face detection method in a cockpit provided by an embodiment of the present specification is shown in the following figure:

[0042] Figure 3 A structural schematic diagram of a target detection model training device based on knowledge distillation provided by an embodiment of the present specification is shown in the following figure:

[0043] Figure 4 A structural schematic diagram of a face detection device in a cockpit provided by an embodiment of the present specification is shown in the following figure:

[0044] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present specification is shown in the following figure. DETAILED DESCRIPTION

[0045] Embodiments of the present specification provide a face detection method in a cockpit, a target detection model training method and device, and the general idea is as follows:

[0046] The confidence of the teacher network and the student network outputting a prediction box for the same target object in the current batch of sample images is determined, and the loss weight of the regression branch corresponding to the target object is determined according to the confidence of the teacher network and the student network outputting a prediction box for the same target object, and the loss value of the regression loss function is adjusted according to the loss weight of the regression branch corresponding to each target object in the current batch of sample images, and the regression loss function is used to train the student network based on knowledge distillation. Through the embodiments of the present application, the prediction accuracy of the target detection model trained based on knowledge distillation can be improved.

[0047] In order to better understand the above technical solutions, the technical solutions of the embodiments of the present specification will be described in detail below through the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present specification and the embodiments are detailed descriptions of the technical solutions of the embodiments of the present specification, and are not limitations of the technical solutions of the present specification. In the case of no conflict, the technical features in the embodiments of the present specification and the embodiments can be combined with each other.

[0048] The target detection model training based on knowledge distillation includes: 1, knowledge distillation at the level of Backbone; 2, knowledge distillation at the level of RPN (region proposal network); 3, knowledge distillation at the level of RCN (region classification network).

[0049] The knowledge distillation at the Backbone level can use a feature transfer method, and an Adaptation layer is used to upgrade / diminish the features of the student network, so that the feature dimensionality is the same as that of the teacher network. The knowledge distillation at the RPN level and the knowledge distillation at the RCN level are both for the classification task and the regression task. The method for training a target detection model based on knowledge distillation provided in the embodiments of the present specification is applied to the knowledge distillation of the regression task, and the knowledge distillation of the other aspects described above can refer to the prior art, which is not limited herein.

[0050] In a first aspect, the embodiments of the present specification provide a method for training a target detection model based on knowledge distillation, which is applied to the knowledge distillation of the regression task, and the method for training a target detection model based on knowledge distillation is as shown in Figure 1 The method for training a target detection model based on knowledge distillation specifically includes the following steps:

[0051] S101, determining the confidence of the prediction box output by the teacher network and the student network for the same target object in the current batch of sample images.

[0052] The first sample image set is divided into multiple batches of sample images, and a batch of sample images is a batch. The training of the student network based on knowledge distillation is performed in batches. Each batch of sample images includes multiple sample images, and each sample image includes at least one object to be detected. A target object is a target object. The teacher network and the student network are used to perform forward prediction on each sample image in the current batch of sample images, and the confidence of the prediction box output by the teacher network and the student network for the same target object in the same sample image is obtained.

[0053] The target object can be a face region in the sample image or a specific image region that needs to be detected. In the case of training a face detection model using a teacher network and a student network, there is at least one face region on each sample image.

[0054] Since there is one or more target objects on each sample image, forward prediction of a sample image by the teacher network will correspondingly output one or more prediction boxes, and forward prediction of a sample image by the student network will also correspondingly output one or more prediction boxes. For example, there are three faces on a sample image, and forward prediction of the sample image by the teacher network and the student network will output three prediction boxes. The prediction box is a Prediction box, which is the box calculated and output by the teacher network and the student network. In contrast, the true box is a Ground truth box, which is the position manually labeled on the sample image and stored in the label file.

[0055] Before step S101, a complex network model and a simple network model need to be constructed in advance; the simple network model is taken as a student network, and the complex network model is pre-trained by using a second sample image set to generate a teacher network, wherein the prediction accuracy of the teacher network is greater than that of the student network. It should be noted that the second sample image set can be the same as or different from the first sample image set.

[0056] S102, determining, according to the confidence of the prediction frame output by the teacher network and the student network for the same target object, the loss weight of the target object corresponding to the regression branch.

[0057] In the embodiments of the present specification, the confidence of the prediction frame output by the teacher network and the student network for the same target object comprises: a first confidence score of the prediction frame output by the teacher network for the target object, and a second confidence score of the prediction frame output by the student network for the target object.

[0058] For the same target object on the same sample image, the first confidence score of the prediction frame output by the teacher network for the target object is obtained by forward calculation of the teacher network, and the second confidence score of the prediction frame output by the student network for the target object is obtained by forward calculation of the student network; the loss weight of the target object corresponding to the regression branch is obtained according to the first confidence score and the second confidence score.

[0059] It should be understood that the current batch of sample images correspond to M*N regression branches, and one target object corresponds to one regression branch, and the loss weight corresponding to each regression branch in the M*N regression branches needs to be adjusted, wherein M is the sample quantity of the current batch of sample images, and N is the number of target objects existing in one sample image.

[0060] In some embodiments, obtaining the loss weight corresponding to the target object according to the first confidence score and the second confidence score comprises: determining a confidence ratio of the first confidence score and the second confidence score; and according to the size relationship between the confidence ratio and a reference weight threshold, the loss weight of the target object corresponding to the regression branch is constrained within a preset weight range.

[0061] Specifically, the teacher network is forward calculated to obtain the first confidence score of the prediction frame output by the teacher network for the target object; the student network is forward calculated to obtain the second confidence score of the prediction frame output by the student network for the target object; the first confidence score is divided by the second confidence score to obtain a confidence ratio; and the confidence ratio is processed by the following function to constrain the loss weight of the target object corresponding to the regression branch within a preset weight range:

[0062]

[0063] wherein Rst is the first confidence score, Rs is the second confidence score, and Rst / Rs is the confidence ratio, is the calculated loss weight of the target object for the regression branch, [0, m] is the preset weight range, and the function of the lip function is to process the confidence ratio Rst / Rs to constrain the loss weight of the target object for the regression branch to be between [0, m]: if the confidence ratio Rst / Rs is greater than the reference weight threshold m, the reference weight threshold m is taken as the loss weight of the target object for the regression branch; if the confidence ratio is not greater than the reference weight threshold m, the confidence ratio Rst / Rs is taken as the loss weight of the target object for the regression branch.

[0064] In the specific implementation process, the reference weight threshold m can be set to 3-7. Taking m=3 as an example, if the confidence ratio Rst / Rs calculated for a certain target object is equal to 1.9, then the loss weight weight of the target object for the regression branch is Rst / Rs=1.9; if the confidence ratio Rst / Rs calculated is equal to 3.1, then weight=m=2.

[0065] S103, adjusting the loss value of the regression loss function according to the loss weight of each target object in the current batch of sample images for the regression branch, wherein the regression loss function is used to train the student network based on knowledge distillation.

[0066] For any one target object, if the loss weight weight of the target object is greater than 1, it means that the learning of the student network for the target object in the sample image is not as good as that of the teacher network, and the loss value of the target object for the regression branch needs to be increased; if the loss weight weight of the target object is less than 1, it means that the learning of the student network for the target object is better than that of the teacher network, and the loss value of the target object for the regression branch needs to be reduced.

[0067] Therefore, for each target object on each sample image in the current batch of sample images, the loss value of the target object for the regression branch is adjusted according to the loss weight of the target object for the regression branch. Specifically, the original loss value of the target object for the regression branch can be calculated by weighting according to the loss weight corresponding to the target object, and the adjusted loss value of the target object for the regression branch is specifically referred to as the following formula:

[0068]

[0069] wherein, is the original loss value of the regression branch corresponding to the i-th target object in the current batch of sample images of the student network, is the loss weight of the regression branch corresponding to the i-th target object in the current batch of sample images, is the adjusted loss value of the regression branch of the student network corresponding to the i-th target object in the current batch of sample images.

[0070] It should be noted that the loss function used to calculate the original loss value of each regression branch varies depending on the student network structure, so there is no specific limitation on the specific form of the loss function used to calculate the original loss value of each regression branch. Specifically, loss functions such as mean squared error (the sum of the average distances between the true bounding box and the predicted bounding box) and mean absolute error (the sum of the absolute differences between the true bounding box and the predicted bounding box) can be used to calculate the original loss value of each regression branch. Through the above technical solution, the original loss value of the regression branch corresponding to each target object in the current batch of sample images can be dynamically adjusted to obtain an adjusted loss value.

[0071] After dynamically adjusting the original loss value of the regression branch corresponding to each target object in the current batch of sample images, the adjusted loss value of the regression branch corresponding to each target object in the current batch of sample images is added up, and the current loss value of the regression loss function is determined based on the sum calculation result.

[0072]

[0073] in, is the adjusted loss value of the regression branch corresponding to the i-th target object in the current batch of sample images, is the current loss value of the regression loss function, K=M*N, and K is the number of prediction boxes output by the student network for the current batch of sample images.

[0074] Whether the regression loss function meets the convergence condition is determined according to the current loss value of the regression loss function. If the convergence condition is met, the training of the student network based on knowledge distillation is terminated to obtain the target detection model after knowledge distillation; otherwise, the next batch of sample images are used to continue the training of the student network based on knowledge distillation.

[0075] The above technical solution dynamically adjusts the loss weight of the regression branch corresponding to each object, and the loss weight participates in the calculation of the regression loss function, so that the calculation result of the regression loss function fully reflects the prediction quality of the teacher network and the student network for each object in each sample image. As a result, the prediction accuracy of the target detection model trained based on knowledge distillation can be improved.

[0076] It should be understood that the knowledge distillation-based target detection model training method provided by the embodiments of the present specification needs to be executed on a device with huge computing resources, and the target detection model obtained by training can be applied to a computing resource-limited scene, such as an intelligent terminal, an intelligent driving cabin of a vehicle, and the like.

[0077] If the target detection model after knowledge distillation is a face detection model, it can be applied to the intelligent driving cabin of a vehicle. Therefore, based on the same inventive concept, the embodiments of the present specification also provide a face detection method in a driving cabin, which is applied to a vehicle.

[0078] Reference Figure 2 As shown in the figure, the face detection method in the driving cabin provided by the embodiments of the present application includes the following steps S201-S202:

[0079] S201: performing face detection in the driving cabin of the vehicle by using a face detection model, wherein the face detection model is obtained by training the knowledge distillation-based target detection model training method of any one of the first aspect.

[0080] Since the knowledge distillation-based target detection model training method has been described in detail in the foregoing, more implementation details about this part can be referred to the foregoing, and will not be described here.

[0081] S202: performing one or more of the following tasks for the driver according to the face detection result: a face recognition task, a face key point detection task, a live detection task, and a fatigue monitoring task.

[0082] Performing face detection in the driving cabin of the vehicle by using a face detection model, obtaining a face prediction box.

[0083] In the specific implementation process, performing a face recognition task for the driver according to the face detection result can be: identifying whether the person currently sitting in the driving position is the target driver according to the face prediction box obtained by face detection, so as to facilitate providing a service item corresponding to the target driver. Performing a face key point detection task for the driver according to the face prediction box obtained by face detection can be: locating the key area position of the face from the face prediction box, including: eyebrows, eyes, nose, mouth, face contour, and the like. Performing a live detection task for the driver according to the face prediction box obtained by face detection can be: verifying the identity of the person currently sitting in the driving position, so as to more accurately identify whether the person currently sitting in the driving position is the target driver. Performing a fatigue monitoring task for the driver according to the face prediction box can be: locating the face feature points (such as: eye area, head posture, and the like) from the face prediction box, and predicting whether the driver is fatigued according to the face feature points.

[0084] Since the face detection model used in the face detection method in the cockpit is trained by using the foregoing target detection model training method based on knowledge distillation, the prediction accuracy of the face detection model is higher, the accuracy of face detection in the cockpit can be improved, and thus the face recognition task, the face key point detection task, the live detection task, and the fatigue monitoring task for the driver can be better completed.

[0085] Based on the same inventive concept, the embodiments of the present specification provide a target detection model training device based on knowledge distillation, which is shown in Figure 3 and includes

[0086] a confidence determination unit 301 configured to determine the confidence of the prediction box output by the teacher network and the student network for the same target object in the current batch sample image;

[0087] a weight adjustment unit 302 configured to determine the loss weight of the regression branch corresponding to the target object according to the confidence of the prediction box output by the teacher network and the student network for the same target object;

[0088] a loss calculation unit 303 configured to adjust the loss value of the regression loss function according to the loss weight of the regression branch corresponding to each target object in the current batch sample image, the regression loss function being used to train the student network based on knowledge distillation.

[0089] In some embodiments, the weight adjustment unit 302 includes: an object determination unit configured to respectively take each to-be-detected object in the current batch sample image as a target object, the current batch sample image including a plurality of sample images, and each sample image including at least one to-be-detected object; a first calculation unit configured to perform forward calculation through the teacher network to obtain a first confidence score of the prediction box output by the teacher network for the target object; a second calculation unit configured to perform forward calculation through the student network to obtain a second confidence score of the prediction box output by the student network for the target object; and a weight calculation unit configured to obtain the loss weight of the regression branch corresponding to the target object according to the first confidence score and the second confidence score.

[0090] In some embodiments, the weight calculation unit is specifically configured to: determine a confidence ratio of the first confidence score and the second confidence score; and constrain the loss weight of the regression branch corresponding to the target object within a preset weight range according to the size relationship between the confidence ratio and a reference weight threshold.

[0091] In some embodiments, the reference weight threshold is set to 3-7.

[0092] In some embodiments, the loss calculation unit 303 is specifically configured to: for each target object in the current batch of sample images, weight the original loss value of the target object for the regression branch according to the loss weight of the target object for the regression branch to obtain an adjusted loss value of the target object for the regression branch; and add the adjusted loss values of each target object for the regression branch in the current batch of sample images to obtain the current loss value of the regression loss function according to the addition result.

[0093] In some embodiments, the device further comprises a training end judgment unit configured to determine whether the regression loss function meets a convergence condition according to the current loss value; if the convergence condition is met, the training of the student network based on knowledge distillation is ended to obtain the target detection model after knowledge distillation, otherwise, the next batch of sample images is used to continue the training of the student network based on knowledge distillation.

[0094] In some embodiments, the device further comprises a pre-training unit configured to: pre-construct a complex network model and a simple network model; pre-train the complex network model to generate the teacher network; and use the simple network model as the student network, wherein the prediction accuracy of the teacher network is higher than that of the student network.

[0095] As to the above-mentioned device for training a target detection model based on knowledge distillation, the specific functions of each unit have been described in detail in the embodiments of the method for training a target detection model based on knowledge distillation provided by the embodiments of the present disclosure, and will not be described in detail here.

[0096] Based on the same inventive concept, the embodiments of the present disclosure provide a device for detecting a face in a cockpit, which is applied to a vehicle. As shown in Figure 4 The device for detecting a face in a cockpit provided by the embodiments of the present disclosure comprises:

[0097] The face detection unit 401 is configured to detect a face in the cockpit of the vehicle by using a face detection model, wherein the face detection model is obtained by training the method for training a target detection model based on knowledge distillation.

[0098] The task execution unit 402 is configured to perform one or more of the following tasks for the driver according to the face detection result: a face recognition task, a face key point detection task, a living body detection task, and a fatigue monitoring task.

[0099] As to the above-mentioned device, the specific functions of each unit have been described in detail in the embodiments of the method for detecting a face in a cockpit provided by the embodiments of the present disclosure, and will not be described in detail here.

[0100] Based on the same inventive concept, the embodiments of the present specification also provide an electronic device, such as Figure 5 As shown in FIG. 5, the electronic device includes a memory 504, a processor 502, and a computer program stored in the memory 504 and executable on the processor 502, and the processor 502 implements the steps of any of the embodiments of the knowledge distillation-based target detection model training method as described above or the steps of any of the embodiments of the cockpit face detection method when executing the program.

[0101] In the above embodiments, the steps of the knowledge distillation-based target detection model training method or the steps of the cockpit face detection method can be implemented by the electronic device shown in FIG. 5. Figure 5 In the above embodiments, the steps of the knowledge distillation-based target detection model training method or the steps of the cockpit face detection method can be implemented by the electronic device shown in FIG. 5.

[0102] In the above embodiments, the steps of the knowledge distillation-based target detection model training method or the steps of the cockpit face detection method can be implemented by the electronic device shown in FIG. 5.

[0103] The embodiments of the present specification are described with reference to flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks

[0104] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0106] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims shall cover all such modifications and variations as fall within the true spirit and scope of the present application.

[0107] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the present application can be practiced otherwise than as specifically described.

Claims

1. A method for training an object detection model based on knowledge distillation, characterized in that: include: Determine the confidence of the prediction boxes output by the teacher network and the student network for the same target object in the current batch of sample images; According to the confidence of the prediction boxes output by the teacher network and the student network for the same target object, the loss weight of the regression branch corresponding to the target object is determined, including: taking each object to be detected in the current batch of sample images as the target object, the current batch of sample images includes multiple sample images, and there is at least one object to be detected in each sample image; performing forward calculation through the teacher network to obtain a first confidence score of the prediction box output by the teacher network for the target object; performing forward calculation through the student network to obtain a second confidence score of the prediction box output by the student network for the target object; obtaining the loss weight of the regression branch corresponding to the target object according to the first confidence score and the second confidence score; wherein, obtaining the loss weight of the regression branch corresponding to the target object according to the first confidence score and the second confidence score includes: determining a confidence ratio of the first confidence score to the second confidence score; and constraining the loss weight of the regression branch corresponding to the target object within a preset weight range according to the relationship between the confidence ratio and a reference weight threshold; According to the loss weight of the regression branch corresponding to each target object in the current batch of sample images, the loss value of the regression loss function is adjusted, and the regression loss function is used to train the student network based on knowledge distillation.

2. The method according to claim 1, wherein The reference weight threshold is set to 3-7.

3. The method according to claim 1, wherein The step of adjusting the loss value of the regression loss function according to the loss weight of the regression branch corresponding to each target object in the current batch of sample images includes: For each target object in the current batch of sample images, the original loss value of the regression branch corresponding to the target object is weighted according to the loss weight of the regression branch corresponding to the target object to obtain the adjusted loss value of the regression branch corresponding to the target object; The adjusted loss values ​​of the regression branches corresponding to each target object in the current batch of sample images are summed up, and the current loss value of the regression loss function is determined according to the summed up result.

4. The method according to claim 1 or 3, wherein: After determining the current loss value of the regression loss function according to the sum calculation result, the method further includes: Determine whether the regression loss function meets the convergence condition according to the current loss value; If the convergence condition is met, the training of the student network based on knowledge distillation is terminated, and the target detection model after knowledge distillation is obtained; otherwise, the next batch of sample images is used to continue the training of the student network based on knowledge distillation.

5. The method according to claim 1, wherein Also includes: Pre-build complex network models and simple network models; Pre-training the complex network model to generate the teacher network; The simple network model is used as the student network, wherein the prediction accuracy of the teacher network is higher than the prediction accuracy of the student network.

6. A method for detecting human faces in a cockpit, applied to a vehicle, characterized in that: The method comprises: Performing face detection in the vehicle cockpit using a face detection model, wherein the face detection model is trained by the object detection model training method based on knowledge distillation according to any one of claims 1 to 5; According to the face detection results, one or more of the following tasks are performed for the driver: face recognition task, face key point detection task, liveness detection task and fatigue monitoring task.

7. A target detection model training device based on knowledge distillation, characterized in that: include: A confidence determination unit, used to determine the confidence of the prediction boxes output by the teacher network and the student network for the same target object in the current batch of sample images; A weight adjustment unit is used to determine the loss weight of the regression branch corresponding to the target object based on the confidence of the prediction boxes output by the teacher network and the student network for the same target object, including: taking each object to be detected in the current batch of sample images as the target object, the current batch of sample images includes multiple sample images, and each sample image contains at least one object to be detected; performing forward calculation through the teacher network to obtain a first confidence score of the prediction box output by the teacher network for the target object; performing forward calculation through the student network to obtain a second confidence score of the prediction box output by the student network for the target object; obtaining the loss weight of the regression branch corresponding to the target object based on the first confidence score and the second confidence score; wherein, obtaining the loss weight of the regression branch corresponding to the target object based on the first confidence score and the second confidence score includes: determining a confidence ratio of the first confidence score to the second confidence score; and constraining the loss weight of the regression branch corresponding to the target object within a preset weight range based on the relationship between the confidence ratio and a reference weight threshold; A loss calculation unit is used to adjust the loss value of the regression loss function according to the loss weight of the regression branch corresponding to each target object in the current batch of sample images, and the regression loss function is used to train the student network based on knowledge distillation.

8. A face detection device in a cockpit, used in a vehicle, characterized in that: The device comprises: a face detection unit, configured to detect faces in the vehicle cockpit using a face detection model, wherein the face detection model is trained using the object detection model training method based on knowledge distillation according to any one of claims 1 to 5; The task execution unit is used to execute one or more of the following tasks for the driver according to the face detection result: face recognition task, face key point detection task, liveness detection task and fatigue monitoring task.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented, or the steps of the method according to claim 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the steps of the method according to any one of claims 1 to 5, or implements the steps of the method according to claim 6.

Images