Face detection model training method and device, face detection method and device, equipment and medium

Abstract

The embodiment of the invention provides a face detection model training method and device, a face detection method and device, equipment and a medium, and the model training method comprises the steps: inputting each training image into an initial face detection model, and obtaining a prediction label; determining a positive sample and a negative sample in the training image according to the difference between the prediction label and the calibration label corresponding to each region included in the training image and the relationship between the rotation angle of the human face in the training image and a preset angle threshold range; calculating the face detection loss based on the difference between the prediction label and the calibration label corresponding to the positive sample and the difference between the prediction label and the calibration label corresponding to the negative sample; and training the initial face detection based on the face detection loss to obtain a face detection model. The human face detection model is used for human face detection, and the human face detection result is output when the rotation angle of the human face is within the preset angle threshold range, so that human face false detection and missing detection in the image can be reduced, and the human face detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a training method, a detection method, a device, equipment and a medium for a face detection model. Background Art

[0002] Face detection is a hot topic and a difficult topic in the field of computer vision and pattern recognition. Face detection refers to the process of searching an image using a certain strategy to determine whether the image contains a face. If the image contains a face, the position, size, and posture of the face are obtained. Face detection has been widely used in the fields of intelligent transportation, identity recognition, and emotion analysis.

[0003] In the face detection scenario, the face in the image may be rotated, that is, the face in the image has a certain rotation angle and is not in a vertical state, that is, the line connecting the midpoints of the two eyes of the face in the image is not parallel to the horizontal plane. For example, the face in the ID photo uploaded by the user may be upside down, etc. When using the current face detection model to perform face detection on such images, it is easy to make false detections and missed detections of faces, and the face detection accuracy is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a training method, a detection method, an apparatus, a device and a medium for a face detection model to reduce false detection and missed detection of faces in images and improve the accuracy of face detection. The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for training a face detection model, the method comprising:

[0006] Obtain multiple training images and calibration labels corresponding to each training image;

[0007] Input each training image into an initial face detection model to obtain a prediction label corresponding to each area included in each training image;

[0008] For each training image, determine the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0009] Calculating the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample and the difference between the predicted label and the calibrated label corresponding to the negative sample;

[0010] Based on the face detection loss, the parameters of the initial face detection model are adjusted until the initial face detection model meets the convergence condition, so as to obtain the face detection model, wherein the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

[0011] Optionally, the step of determining positive samples and negative samples in the training image according to the difference between the predicted labels and the calibrated labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range includes:

[0012] Based on the center point position of the calibration label corresponding to the training image, determining a positive sample candidate region from each region included in the training image;

[0013] Determine the number of positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0014] For each positive sample candidate region, based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region, the target loss corresponding to the positive sample candidate region is calculated, wherein the target loss includes position prior loss, sample regression loss and sample classification loss;

[0015] Based on the target loss corresponding to each positive sample candidate region, determining the number of positive sample candidate regions from the positive sample candidate regions as positive samples;

[0016] Other regions in the training image except the positive samples are determined as negative samples.

[0017] Optionally, the step of determining the number of positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region and the relationship between the rotation angle of the face in the training image and the preset angle threshold range includes:

[0018] For each positive sample candidate region, calculate the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate region;

[0019] Summing a maximum preset number of intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result;

[0020] The sum result is rounded to an integer to obtain the number of candidate positive samples corresponding to the calibration label;

[0021] Based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, the number of positive samples in the training image is determined.

[0022] Optionally, the step of determining the number of positive samples of the positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and a preset angle threshold range includes:

[0023] Based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule, the quantity coefficient corresponding to the training image is calculated, wherein the preset coefficient determination rule is such that: when the rotation angle of the face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range and greater than the preset angle threshold range, and the first preset value is greater than the second preset value and greater than 0;

[0024] The product of the quantity coefficient and the number of candidate positive samples is determined as the number of positive samples in the training image.

[0025] Optionally, the step of calculating the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and a preset coefficient determination rule includes:

[0026] Based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula, the quantity coefficient corresponding to the training image is calculated;

[0027]

[0028] Among them, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range.

[0029] Optionally, the step of calculating the target loss corresponding to the positive sample candidate region based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region includes:

[0030] Obtain the confidence level corresponding to the positive sample candidate region;

[0031] According to the center point position coordinates of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label, the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position of the calibration label is calculated according to the following formula;

[0032]

[0033] Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label;

[0034] Based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, the loss of the positive sample candidate region is calculated according to the following formula, wherein the loss includes sample regression loss and sample classification loss;

[0035] cost iou，conf =iou α +conf β

[0036] Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold;

[0037] Based on the prior center distance and the loss, the target loss corresponding to the positive sample candidate region is calculated according to the following formula:

[0038] cost = f(x)cost iou，conf +(1-f(x))dist(gt,pos box )

[0039] Wherein, f(x) is the quantitative coefficient.

[0040] Optionally, the step of determining the number of positive sample candidate regions as positive samples from the positive sample candidate regions based on the target loss corresponding to each positive sample candidate region includes:

[0041] Based on the target loss corresponding to each positive sample candidate region, the positive sample candidate region corresponding to the target loss with the smallest number of positive samples in the target loss is taken as a positive sample.

[0042] In a second aspect, an embodiment of the present application provides a face detection method, the method comprising:

[0043] Acquire the image to be detected;

[0044] Inputting the image to be detected into a pre-trained face detection model, and determining whether the face detection model outputs a face detection result, wherein the face detection model is pre-trained according to any one of the methods described in the first aspect above;

[0045] If the face detection model does not output a face detection result, rotating the image to be detected according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected, wherein the preset angle is determined based on the preset angle threshold range;

[0046] Returning to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state;

[0047] If the face detection model outputs a face detection result, it is determined that the face detection result is the face detection result of the image to be detected.

[0048] Optionally, the preset angle enables the rotation angle of the face in the image to be detected to fall within the preset angle threshold range at least once during the rotation of the image to be detected.

[0049] In a third aspect, an embodiment of the present application provides a training device for a face detection model, the device comprising:

[0050] A first acquisition module is used to acquire a plurality of training images and a calibration label corresponding to each training image;

[0051] A second acquisition module is used to input each training image into an initial face detection model to obtain a prediction label corresponding to each area included in each training image;

[0052] A positive and negative sample determination module is used to determine, for each training image, the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0053] A face detection loss calculation module, used to calculate the face detection loss based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample;

[0054] A third acquisition module is used to adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence condition, thereby obtaining the face detection model, wherein the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

[0055] Optionally, the positive and negative sample determination module includes:

[0056] A positive sample candidate region determining unit, configured to determine a positive sample candidate region from each region included in the training image based on the center point position of the calibration label corresponding to the training image;

[0057] A positive sample quantity determination unit, used to determine the positive sample quantity of the positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0058] A target loss calculation unit is used to calculate, for each positive sample candidate region, a target loss corresponding to the positive sample candidate region based on a difference between a predicted label and a calibrated label corresponding to the positive sample candidate region, wherein the target loss includes a position prior loss, a sample regression loss, and a sample classification loss;

[0059] A positive sample determining unit, configured to determine the number of positive sample candidate regions from the positive sample candidate regions as positive samples based on the target loss corresponding to each positive sample candidate region;

[0060] The negative sample determination unit is used to determine other areas in the training image except the positive samples as negative samples.

[0061] Optionally, the positive sample quantity determination unit includes:

[0062] An intersection-over-union ratio determination subunit is used to calculate, for each positive sample candidate region, an intersection-over-union ratio between a predicted label and a calibrated label corresponding to the positive sample candidate region;

[0063] A summation result acquisition subunit is used to sum a maximum preset number of intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result;

[0064] A candidate positive sample quantity acquisition subunit is used to round the sum result as the candidate positive sample quantity corresponding to the calibration label;

[0065] The positive sample quantity determination subunit is used to determine the positive sample quantity of the positive samples in the training image based on the candidate positive sample quantity and the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0066] Optionally, the positive sample quantity determination subunit includes:

[0067] A quantity coefficient calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the human face in the training image and the preset angle threshold range and the preset coefficient determination rule, wherein the preset coefficient determination rule is such that: when the rotation angle of the human face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the human face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the human face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range and greater than the preset angle threshold range, and the first preset value is greater than the second preset value and greater than 0;

[0068] The first determining subunit is used to determine the product of the quantity coefficient and the number of candidate positive samples as the number of positive samples in the training image.

[0069] Optionally, the quantity coefficient calculation subunit includes:

[0070] A first calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula;

[0071]

[0072] Among them, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range.

[0073] Optionally, the target loss calculation unit includes:

[0074] A confidence acquisition subunit, used to obtain the confidence corresponding to the positive sample candidate region;

[0075] The prior center distance calculation subunit is used to calculate the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label according to the following formula;

[0076]

[0077] Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label;

[0078] A loss calculation subunit, configured to calculate the loss of the positive sample candidate region according to the following formula based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, wherein the loss includes sample regression loss and sample classification loss;

[0079] cost iou，conf =iou α +conf β

[0080] Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold;

[0081] The target loss calculation subunit is used to calculate the target loss corresponding to the positive sample candidate region based on the prior center distance and the loss according to the following formula:

[0082] cost = f(x)cost iou，conf +(1-f(x))dist(gt,pos box )

[0083] Wherein, f(x) is the quantitative coefficient.

[0084] Optionally, the positive sample determining unit includes:

[0085] The positive sample determination subunit is used to take the positive sample candidate region corresponding to the smallest number of positive sample target losses in the target losses as the positive sample based on the target loss corresponding to each positive sample candidate region.

[0086] In a fourth aspect, an embodiment of the present application provides a face detection device, the device comprising:

[0087] An image acquisition module to be detected is used to acquire the image to be detected;

[0088] A detection result judgment module, used for inputting the image to be detected into a pre-trained face detection model to determine whether the face detection model outputs a face detection result, wherein the face detection model is pre-trained according to positive samples and negative samples included in the training image, the positive samples are samples used to predict faces in the training image, the negative samples are samples not used to predict faces in the training image, and the number of the positive samples and the number of the negative samples are determined based on the rotation angle of the face in the training image and a preset angle threshold range;

[0089] A detection image rotation module, if the face detection model does not output a face detection result, rotates the detection image according to a preset direction and a preset angle to obtain a rotated detection image as the detection image, wherein the preset angle is determined based on the preset angle threshold range;

[0090] A detection result acquisition module, used for returning to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state;

[0091] The face detection result determination module is used to determine that the face detection result is the face detection result of the image to be detected if the face detection model outputs the face detection result.

[0092] Optionally, the preset angle enables the rotation angle of the face in the image to be detected to fall within the preset angle threshold range at least once during the rotation of the image to be detected.

[0093] In a fifth aspect, an embodiment of the present application provides an electronic device, including:

[0094] Memory, used to store computer programs;

[0095] The processor is used to implement any of the methods described in the first aspect or the second aspect when executing the program stored in the memory.

[0096] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspect or the second aspect is implemented.

[0097] Beneficial effects of the embodiments of the present application:

[0098] According to the solution provided by the embodiment of the present application, the electronic device can obtain multiple training images and the calibration labels corresponding to each training image; input each training image into the initial face detection model to obtain the predicted labels corresponding to each area included in each training image; for each training image, according to the difference between the predicted labels and the calibration labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, determine the positive sample and the negative sample in the training image; based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample, calculate the face detection loss; based on the face detection loss, adjust the parameters of the initial face detection model until the initial face detection model meets the convergence condition, and obtain the face detection model. The face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and does not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range. Since the face detection model is obtained based on positive and negative sample training, the number of positive samples and negative samples is determined according to the difference between the predicted label and the calibration label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range. For the input image to be detected, when the rotation angle of the face in the image to be detected is within the preset angle threshold range, the face detection model outputs the face detection result, and when the rotation angle of the face in the image to be detected is not within the preset angle threshold range, the face detection model does not output the face detection result. Therefore, when using the face detection model for face detection, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. This can reduce false detection and missed detection of faces in the image and improve the accuracy of face detection. Of course, the implementation of any product or method of the present application does not necessarily require all the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0100] Figure 1 A flowchart of a method for training a face detection model provided in an embodiment of the present application;

[0101] Figure 2 for Figure 1A specific flow chart of step S103 in the illustrated embodiment;

[0102] Figure 3 for Figure 2 A specific flow chart of step S202 in the illustrated embodiment;

[0103] Figure 4 for Figure 3 A specific flow chart of step S304 in the illustrated embodiment;

[0104] Figure 5 A specific flow chart of the face detection model training method provided in the embodiment of the present application;

[0105] Figure 6 A flow chart of a face detection method provided in an embodiment of the present application;

[0106] Figure 7 A specific flow chart of the face detection method provided in the embodiment of the present application;

[0107] Figure 8 A schematic diagram of the structure of a face detection device provided in an embodiment of the present application;

[0108] Fig. 9 A schematic diagram of the structure of a training device for a face detection model provided in an embodiment of the present application;

[0109] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0110] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0111] It should be noted that in the technical solution of this application, the acquisition, storage, use, processing, transmission, provision and disclosure of user personal information involved are all carried out with the authorization of the user. User personal information, for example, the image to be detected in the embodiment of this application, etc.

[0112] In order to reduce false detection and missed detection of faces in images and improve the accuracy of face detection, the embodiments of the present application provide a training method for a face detection model, a face detection method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The following first introduces a training method for a face detection model provided in the embodiments of the present application.

[0113] A method for training a face detection model provided in an embodiment of the present application can be applied to any device that needs to train a face detection model, for example, a laptop computer, a server, an image processor, a tablet computer, a mobile phone, etc., which is not specifically limited here. For the sake of clarity of description, it is referred to as an electronic device below.

[0114] like Figure 1 As shown, a training method for a face detection model comprises:

[0115] S101, obtaining a plurality of training images and a calibration label corresponding to each training image;

[0116] S102, inputting each training image into an initial face detection model to obtain a prediction label corresponding to each region included in each training image;

[0117] S103, for each training image, determining positive samples and negative samples in the training image according to the difference between the predicted labels and the calibrated labels corresponding to each region included in the training image and the relationship between the rotation angle of the face in the training image and a preset angle threshold range;

[0118] S104, calculating the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample, and the difference between the predicted label and the calibrated label corresponding to the negative sample;

[0119] S105, adjusting parameters of the initial face detection model based on the face detection loss until the initial face detection model meets a convergence condition, thereby obtaining the face detection model.

[0120] Among them, the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

[0121] It can be seen that in the scheme provided in the embodiment of the present application, the electronic device can obtain multiple training images and the calibration labels corresponding to each training image; input each training image into the initial face detection model to obtain the predicted labels corresponding to each area included in each training image; for each training image, according to the difference between the predicted labels and the calibration labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, determine the positive sample and the negative sample in the training image; based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample, calculate the face detection loss; based on the face detection loss, adjust the parameters of the initial face detection model until the initial face detection model meets the convergence condition, and obtain the face detection model. Among them, the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range. Since the face detection model is obtained based on positive and negative sample training, the number of positive samples and negative samples is determined according to the difference between the predicted label and the calibration label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range. For the input image to be detected, when the rotation angle of the face in the image to be detected is within the preset angle threshold range, the face detection model outputs the face detection result, and when the rotation angle of the face in the image to be detected is not within the preset angle threshold range, the face detection model does not output the face detection result. Therefore, when using the face detection model for face detection, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. This can reduce false detection and missed detection of faces in the image and improve the accuracy of face detection.

[0122] In the process of training the face detection model, the electronic device can obtain multiple training images and calibration labels corresponding to each training image, that is, execute step S101. There is a rotated face in each training image, and there may be one or more rotated faces. The calibration label is a true value label, which indicates the real area where the face exists in the training image.

[0123] Then, in the above step S102, the electronic device can input each training image into the initial face detection model to obtain the prediction labels corresponding to each area included in each training image. Since the pre-trained face detection model can output faces whose rotation angles fall within the preset angle threshold range, the face detection model can be trained based on the face areas with rotation angles and the areas without faces in each training image.

[0124] After acquiring multiple training images, the electronic device inputs each training image into the initial face detection model. The initial face detection model can predict each area included in each training image and output the prediction label corresponding to each area. The electronic device can obtain the prediction label corresponding to each area included in each training image. Among them, the area included in the image to be detected can be set according to actual conditions, such as a 10x10 area, an 8x8 area, etc., which is not specifically limited here. The prediction label can indicate whether each area is predicted to be a face area.

[0125] For example, the electronic device obtains n training images, namely P1, P2...Pn, and the electronic device can input P1, P2...Pn into the initial face detection model respectively. Among them, the area included in P1 is a 5x5 area. When the electronic device inputs P1 into the initial face detection model, the prediction label corresponding to the 5x5 area included in P1 can be obtained, that is, 25 prediction labels can be obtained.

[0126] When the initial face detection model processes the training image, it can also calculate the confidence of each area included in the training image, so the electronic device can obtain the confidence of each area included in the training image. The confidence can represent the probability of a face.

[0127] For each training image, there is a difference between the predicted label and the calibrated label corresponding to each area included in each training image, and the relationship between the rotation angle of the face in each training image and the preset angle threshold range is also different. In one embodiment, for each training image, the electronic device can determine the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0128] Among them, positive samples can enable the trained face detection model to have the ability to determine the foreground (i.e., face), which not only enables the trained face detection model to recognize the foreground target in the image to be detected, but also requires the specific position of the foreground target to be represented by a frame. Negative samples can enable the trained face detection model to have the ability to distinguish the background (i.e., non-face), so that the trained face detection model will not produce false detection of the background.

[0129] For example, an electronic device obtains a training image P1, where P1 includes a 5x5 area, there are differences between the predicted labels and the corresponding calibration labels in some areas, the rotation angle of the face in P1 is 60°, and the preset angle threshold range is [0, 45°). Then the electronic device can determine the positive samples and negative samples in the training image based on the differences between the predicted labels and the calibration labels corresponding to the areas included in the training image and the relationship between the rotation angle of the face 60° and the preset angle threshold range [0, 45°).

[0130] Then, the electronic device may execute the above step S104, that is, calculate the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample, and the difference between the predicted label and the calibrated label corresponding to the negative sample.

[0131] After determining the positive sample and the negative sample, the electronic device can calculate the face detection loss using a preset loss function based on the difference between the predicted label and the calibrated label corresponding to the positive sample, and the difference between the predicted label and the calibrated label corresponding to the negative sample, and then determine the difference between the predicted value and the true value. The preset loss function can be a regression loss function, a classification loss function, etc., which are not specifically limited here.

[0132] The loss function can measure the gap between the model's output and the real data. The higher the value of the loss function, the greater the gap between the predicted value and the real value. The lower the value of the loss function, the smaller the gap between the predicted value and the real value, that is, the closer to the real value. Therefore, the face detection loss can represent the difference between the predicted value and the real value of the positive sample and the difference between the predicted value and the real value of the negative sample.

[0133] If the face detection loss is large, it means that the determined positive and negative samples are significantly different from the real positive and negative samples. The electronic device can adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence condition, that is, the face detection loss is minimum or meets the preset threshold, and then obtain the face detection model. For example, if the face detection loss is 2, which is greater than the preset threshold of 0.5, the electronic device can adjust the parameters of the initial face detection model until the face detection loss is no greater than 0.5, complete the training of the face detection model, and obtain the face detection model.

[0134] It can be seen that in this embodiment, the electronic device can determine the positive samples and negative samples in the training image based on the difference between the predicted labels and the calibration labels corresponding to each area included in each training image obtained, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, calculate the face detection loss based on the difference between the predicted labels and the calibration labels corresponding to the positive samples, and the difference between the predicted labels and the calibration labels corresponding to the negative samples, and then adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence conditions to obtain the face detection model. In this way, when the image to be detected is input into the face detection model, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range, so as to reduce false detection of faces in the image and improve the accuracy of face detection.

[0135] As an implementation method of the present application, Figure 2 As shown, the step of determining the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range may include:

[0136] S201, determining a positive sample candidate region from each region included in the training image based on the center point position of the calibration label corresponding to the training image;

[0137] In order to determine the positive samples in the training image, the electronic device may first determine the approximate range where the positive samples are located according to the principle of center prior, and then determine the positive samples from within the range. In one embodiment, for each training image, the electronic device may determine the positive sample candidate region from the regions included in the training image based on the center point position of the calibration label corresponding to the training image. The positive sample candidate region is smaller than the region included in the training image.

[0138] For example, if the training image includes a 10x10 area, the electronic device can use the center point of the calibration label corresponding to the training image as the center and determine a 3x3 area near the center from the 10x10 area as the positive sample candidate area. Of course, a 5x5 area, a 2x3 area, etc. can also be determined.

[0139] S202, determining the number of positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0140] After determining the positive sample candidate area, the electronic device needs to further determine the positive samples from the positive sample candidate area. In one embodiment, the electronic device may first determine the number of positive samples of the positive sample, and then determine the number of positive sample areas from the positive sample candidate area as positive samples. The positive samples may be continuous or scattered in the positive sample candidate area.

[0141] Since the positive sample candidate area includes the area where the face is located, there is a certain relationship between the rotation angle of the face and the preset angle threshold range. The larger the rotation angle of the face, the smaller the possibility of belonging to the preset angle threshold range, and the smaller the value of training the training image, so the number of positive samples in the training image determined from the positive sample candidate area can also be reduced accordingly. In one embodiment, the electronic device can determine the number of positive samples in the training image based on the difference between the predicted label and the calibration label corresponding to each positive sample candidate area, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0142] For example, the electronic device determines that the positive sample candidate area is a 5x5 area from the areas included in the training image, and needs to further determine the positive sample from the 5x5 area. The rotation angle of the face in the training image is 70°, and the preset angle threshold range is [0, 60°). The electronic device can determine the number of positive samples of the positive sample in the training image based on the difference between the predicted label and the calibration label corresponding to each area in the 5x5 area and the relationship between the rotation angle of the face 70° and the preset angle threshold range [0, 60°).

[0143] S203, for each positive sample candidate region, based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region, calculating the target loss corresponding to the positive sample candidate region;

[0144] The target loss includes position prior loss, sample regression loss and sample classification loss.

[0145] After the electronic device determines the number of positive samples, it is necessary to determine the area with the number of positive samples from the positive sample candidate area. In one embodiment, for each positive sample candidate area, the electronic device can calculate the target loss corresponding to the positive sample candidate area based on the difference between the predicted label and the calibration label corresponding to the positive sample candidate area. The target loss may include position prior loss, sample regression loss, and sample classification loss.

[0146] As an implementation method, for each positive sample candidate area, the electronic device can calculate the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate area, and obtain the confidence of the positive sample candidate area, and then calculate the target loss corresponding to the positive sample candidate area based on the confidence, intersection-over-union ratio and the prior center distance between the center point position of the predicted label and the center point position of the calibrated label of the positive sample candidate area.

[0147] For example, the electronic device calculates the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate area as iou1, the confidence of the positive sample candidate area is conf1, and the a priori center distance between the center point position of the predicted label and the center point position of the calibrated label is dist, then the target loss corresponding to the positive sample candidate area can be calculated as f(r)cost iou1，conf1 +(1-f(r))dist. Among them, cost iou1，conf1 is the loss calculated based on the intersection-over-union ratio iou1 and the confidence conf1, and f(r) is determined based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0148] S204, determining the number of positive sample candidate regions as positive samples from the positive sample candidate regions based on the target loss corresponding to each positive sample candidate region;

[0149] S205: Determine other regions in the training image except the positive samples as negative samples.

[0150] After calculating the target loss corresponding to each positive sample candidate area, the electronic device can determine the number of positive samples from the positive sample candidate areas as positive samples according to the size of the target loss corresponding to each positive sample candidate area, and then determine other areas in the training image except the positive samples as negative samples.

[0151] The smaller the target loss is, the smaller the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region is. In one embodiment, the electronic device can determine the number of positive sample candidate regions with the smallest target loss from the positive sample candidate regions. For example, if the number of positive samples is 6 and the positive sample candidate region is a 5x5 region, then the electronic device can determine 6 regions from the 5x5 region with the smallest target loss.

[0152] It can be seen that in this embodiment, for each positive sample candidate area, the electronic device can determine the positive sample candidate area from the areas included in the training image, and then determine the number of positive samples of the positive samples in the training image, calculate the target loss corresponding to the positive sample candidate area, and then determine the number of positive sample candidate areas from the positive sample candidate areas based on the target loss corresponding to each positive sample candidate area as positive samples, and determine other areas in the training image other than the positive samples as negative samples. In this way, positive samples and negative samples suitable for training the initial face detection model can be obtained, and then after the initial face detection model is trained based on the positive samples and negative samples, an output that can suppress the face whose rotation angle exceeds the preset angle threshold range is obtained, so as to reduce false detection of faces in images and improve the accuracy of face detection.

[0153] As an implementation method of the present application, Figure 3 As shown, the step of determining the number of positive samples in the training image based on the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region and the relationship between the rotation angle of the face in the training image and the preset angle threshold range may include:

[0154] S301, for each positive sample candidate region, calculating the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate region;

[0155] For each positive sample candidate area, the electronic device may determine the difference between the predicted label and the calibrated label according to the overlap between the predicted label and the calibrated label. In one embodiment, for each positive sample candidate area, the electronic device may calculate the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate area.

[0156] For example, the intersection of the area corresponding to the predicted label corresponding to the positive sample candidate area and the area corresponding to the calibrated label is S1, and the union of the area corresponding to the predicted label corresponding to the positive sample candidate area and the area corresponding to the calibrated label is S2. Then the electronic device can calculate the intersection ratio between the predicted label corresponding to the positive sample candidate area and the calibrated label as S1 / S2.

[0157] S302, summing up a preset number of the largest intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result;

[0158] S303, rounding the summed result to an integer as the number of candidate positive samples corresponding to the calibration label;

[0159] After calculating the intersection-and-union ratio between the predicted label and the calibrated label corresponding to each positive sample candidate area, the electronic device can sort the intersection-and-union ratios in ascending order or in descending order, where the larger the intersection-and-union ratio, the larger the overlapping area between the predicted label and the calibrated label, that is, the closer the predicted label is to the calibrated label. Therefore, the electronic device can determine the number of candidate positive samples corresponding to the calibrated label based on the intersection-and-union ratio.

[0160] In one implementation, the electronic device may sum the largest preset number of intersection-and-union ratios in the intersection-and-union ratios to obtain a summation result. The summation result is rounded up as the number of candidate positive samples corresponding to the calibration label. In general, the summation result is rounded up, but of course, the summation result may also be rounded down. The preset number may be set based on the number of positive sample candidate regions. For example, if the number of positive sample candidate regions is 10, the preset number may be 8, etc.

[0161] For example, the intersection-and-union ratios corresponding to the 3x3 positive sample candidate area are 0.9, 0.8, 0.6, 0.8, 0.7, 0.6, 0.7, 0.5, and 0.6, respectively, and the preset number is 7. The electronic device sorts the intersection-and-union ratios in ascending order, and obtains 0.5, 0.6, 0.6, 0.6, 0.7, 0.7, 0.8, 0.8, and 0.9. The 7 largest intersection-and-union ratios are summed up, and the sum is 5.1. The sum is rounded down to 5, so the number of candidate positive samples corresponding to the calibration label is 5.

[0162] S304: Determine the number of positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and a preset angle threshold range.

[0163] Since the greater the rotation angle of the face, the less likely it is to fall within the preset angle threshold range, and the less valuable it is to train the training image, the number of positive samples in the training image determined from the positive sample candidate region may be reduced accordingly. In one embodiment, the electronic device may determine the number of positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0164] For example, the number of candidate positive samples is 6, the rotation angle of the face in the training image is 60°, and the preset angle threshold range is [0, 45°). The rotation angle of the face in the training image is already greater than the preset angle threshold range, and the training value becomes smaller. The electronic device can determine a quantity coefficient f(r) according to a certain formula based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range, multiply f(r) by the number of candidate positive samples to obtain a multiplication result, and then determine the multiplication result as the number of positive samples.

[0165] It can be seen that in this embodiment, the electronic device can calculate the intersection-and-union ratio between the predicted label and the calibration label corresponding to each positive sample candidate area, sum the largest preset number of intersection-and-union ratios in the intersection-and-union ratios to obtain the sum result, round the sum result as the number of candidate positive samples corresponding to the calibration label, and determine the number of positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range. The number of positive samples obtained in this way is more reasonable.

[0166] As an implementation method of the present application, Figure 4 As shown, the step of determining the number of positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range may include:

[0167] S401, calculating a quantity coefficient corresponding to the training image based on a relationship between a rotation angle of a face in the training image and a preset angle threshold range and a preset coefficient determination rule;

[0168] Among them, the preset coefficient determination rule is as follows: when the rotation angle of the face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range, greater than the preset angle threshold range, and the first preset value is greater than the second preset value, greater than 0.

[0169] S402: Determine the product of the quantity coefficient and the number of candidate positive samples as the number of positive samples in the training image.

[0170] Since the greater the rotation angle of the face, the smaller the possibility of belonging to the preset angle threshold range, certain rules can be set in advance to determine a certain proportional coefficient based on the relationship between the rotation angle of the face and the preset angle threshold range, and then determine the number of positive samples according to the proportional coefficient.

[0171] In one implementation, the electronic device may calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and a preset angle threshold range and a preset coefficient determination rule.

[0172] Among them, the preset coefficient determination rule is as follows: when the rotation angle of the face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range, greater than the preset angle threshold range, and the first preset value is greater than the second preset value, greater than 0.

[0173] That is to say, when the rotation angle of the face is within the preset angle threshold range, the quantitative coefficient is the largest, when the rotation angle of the face is within the first preset range, the quantitative coefficient is between the minimum and the maximum, and when the rotation angle of the face is within the second preset range, the quantitative coefficient is the smallest.

[0174] Then, when the electronic device calculates the quantity coefficient corresponding to the training image, the quantity coefficient can be multiplied by the number of candidate positive samples to obtain a multiplication result, and the multiplication result is determined as the number of positive samples of the positive samples in the training image. In the case where the multiplication result is not an integer, the summation result can be rounded up or down. Among them, the larger the quantity coefficient, the more positive samples the electronic device can determine, and the more positive samples there are, the more beneficial it is to train the initial face detection model.

[0175] For example, the number of candidate positive samples is 6, the rotation angle of the face in the training image is θ / 2, when the rotation angle of the face is within [0, θ), the quantity coefficient is 1, when the rotation angle of the face is within [θ, 90), the quantity coefficient is f(r), and when the rotation angle of the face is within [90, 180], the quantity coefficient is 0. Then the electronic device can calculate that the quantity coefficient corresponding to the training image is 1, and then multiply the quantity coefficient by the number of candidate positive samples by 6 to determine the number of positive samples in the training image.

[0176] It can be seen that in this embodiment, the electronic device can calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule, and determine the product of the quantity coefficient and the number of candidate positive samples as the number of positive samples of the positive samples in the training image. In this way, the relationship between the rotation angle of the face and the preset angle threshold range is introduced as a reference item, so that the number of positive samples obtained is more reasonable.

[0177] As an implementation of an embodiment of the present application, the step of calculating the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule may include:

[0178] Based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula, the quantity coefficient corresponding to the training image is calculated;

[0179]

[0180] Among them, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range.

[0181] In one embodiment, the electronic device can calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the preset formula. The preset formula is: when the rotation angle of the face is within [0, θ), the quantity coefficient is 1; when the rotation angle of the face is within [θ, 90), the quantity coefficient is When the rotation angle of the face is [90, 180], the coefficient is 0.

[0182] Specifically, take the rotation angle of the face in the range of 0-180° as an example, and the same applies to 180-360°. The f(x) function can detect faces with a rotation angle in the range of [0, θ) normally, and the detection rate of faces with a rotation angle in the range of [θ, 90) is partially suppressed. The closer to 90°, the more obvious the suppression effect. If the faces with a rotation angle in the range of [θ, 90) are completely suppressed, the detection rate of faces with a rotation angle in the range of [0, θ) will decrease. Faces with a rotation angle greater than 90° will be completely suppressed.

[0183] For example, the electronic device may calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula;

[0184]

[0185] Wherein, x is the rotation angle of the face, [0, 45) is the preset angle threshold range, [45, 90) is the first preset range, and [90, 180] is the second preset range.

[0186] It can be seen that in this embodiment, the electronic device can calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the preset formula. In this way, the preset coefficient is introduced as the proportional coefficient, so that the number of positive samples obtained is more reasonable.

[0187] As an implementation of an embodiment of the present application, the step of calculating the target loss corresponding to the positive sample candidate region based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region may include:

[0188] Obtain the confidence level corresponding to the positive sample candidate region;

[0189] According to the center point position coordinates of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label, the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position of the calibration label is calculated according to the following formula;

[0190]

[0191] Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label;

[0192] Based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, the loss of the positive sample candidate region is calculated according to the following formula, wherein the loss includes sample regression loss and sample classification loss;

[0193] cost iou，conf =iou α +conf β

[0194] Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold.

[0195] Based on the prior center distance and the loss, the target loss corresponding to the positive sample candidate region is calculated according to the following formula:

[0196] cost = f(x)cost iou，conf +(1-f(x))dist(gt,pos box )

[0197] Wherein, f(x) is the quantitative coefficient.

[0198] Since the initial face detection model calculates the confidence of each area included in the training image while processing the training image, the electronic device can obtain the confidence corresponding to each positive sample candidate area. Then, based on the confidence, intersection-over-union ratio and the prior center distance between the center point position of the predicted label and the center point position of the calibrated label of the positive sample candidate area, the target loss corresponding to the positive sample candidate area is calculated.

[0199] If the rotation angle of the face in the training image is large, the electronic device determines that the number of positive samples in the training image is small, and optimization is relatively difficult. Therefore, dist(gt, pos) can be calculated based on the center distance prior. box ), so that the positive sample candidate area closer to the center of the calibration label corresponding to the training image is more likely to become a positive sample.

[0200] Then, for each positive sample candidate area, the electronic device can calculate the center point position coordinates of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label according to the formula Calculate the a priori center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position of the calibration label. gt Used to represent the center point coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label.

[0201] Next, the electronic device can calculate the confidence level and intersection-over-union ratio of the positive sample candidate region according to the formula cost iou，conf =iou α +conf β Calculate the loss of the positive sample candidate area, where the loss includes sample regression loss and sample classification loss; iou is the intersection-over-union ratio, conf is the confidence, α is the first preset threshold, and β is the second preset threshold.

[0202] After calculating the prior center distance and loss, the electronic device can calculate the cost=f(x)cost based on the prior center distance and loss. iou，conf +(1-f(x))dist(gt,pos box ) calculates the target loss corresponding to the positive sample candidate region, and then determines the positive sample from the positive sample candidate region based on the target loss of each positive sample candidate region. Wherein f(x) is the quantity coefficient.

[0203] It can be seen that in this embodiment, the electronic device can calculate the target loss corresponding to each positive sample candidate area based on the prior center distance and loss after calculating the prior center distance and loss, so as to further determine the positive sample from the positive sample candidate area based on the target loss of each positive sample candidate area.

[0204] As an implementation manner of an embodiment of the present application, the step of determining the number of positive sample candidate regions as positive samples from the positive sample candidate regions based on the target loss corresponding to each positive sample candidate region may include:

[0205] Based on the target loss corresponding to each positive sample candidate region, the positive sample candidate region corresponding to the target loss with the smallest number of positive samples in the target loss is taken as a positive sample.

[0206] After the electronic device obtains the target loss corresponding to each positive sample candidate area, the target losses can be sorted in ascending order or in descending order. The smaller the target loss, the smaller the difference between the predicted label and the calibrated label corresponding to the positive sample candidate area, and the larger the overlapping area. In one embodiment, the electronic device can use the positive sample candidate area corresponding to the target loss with the smallest number of positive samples in the target loss as a positive sample.

[0207] For example, the target losses corresponding to the 3x3 positive sample candidate areas are 0.6, 0.3, 0.5, 0.6, 0.3, 0.2, 0.3, 0.5, 0.1, and 0.2, respectively, and the number of positive samples is 6. The electronic device sorts the target losses in ascending order, and obtains 0.1, 0.2, 0.2, 0.3, 0.3, 0.3, 0.5, 0.5, and 0.6, and takes the positive sample candidate areas corresponding to the smallest 6 target losses as positive samples, that is, the positive sample candidate areas corresponding to 0.1, 0.2, 0.2, 0.3, 0.3, and 0.3 are taken as positive samples.

[0208] It can be seen that in this embodiment, the electronic device can take the positive sample candidate area corresponding to the smallest number of positive samples in the target loss as the positive sample based on the target loss corresponding to each positive sample candidate area. In this way, the predicted label corresponding to the determined positive sample can be close to the calibrated label, which can better train the initial face detection model.

[0209] Figure 5 A specific flow chart of the face detection method provided in the embodiment of the present application. Figure 5 The face detection method provided in the embodiment of the present application is introduced by way of example. Figure 5 As shown, the face detection method provided in the embodiment of the present application may include the following steps:

[0210] S501, obtaining a training image;

[0211] The electronic device can obtain multiple training images and obtain a calibration label corresponding to each training image.

[0212] S502, inputting into an initial face detection model;

[0213] The electronic device may input each training image into an initial face detection model, and the initial face detection model may predict each area included in each training image.

[0214] S503, obtaining a real-time prediction result, obtaining the confidence of each region included in the training image, and calculating an intersection-over-union ratio based on the predicted label and the calibration label corresponding to each region included in the training image;

[0215] The electronic device can obtain the prediction results output by the initial face detection model, that is, obtain the prediction labels corresponding to each area included in the training image, and then calculate the intersection-over-union ratio between the prediction labels corresponding to each area included in the training image and the calibration labels. The electronic device can also obtain the confidence of each area included in the training image calculated by the initial face detection model.

[0216] S504, determining a positive sample candidate region;

[0217] The electronic device can determine the positive sample candidate area from each area included in the training image based on the center point position of the calibration label corresponding to the training image according to the principle of center prior. For example, taking the center point position of the calibration label corresponding to the training image as the center, determine a 3x3 area or a 5x5 area from the vicinity of the center.

[0218] S505, calculating the number of candidate positive samples;

[0219] For each positive sample candidate area, the electronic device can determine the intersection-and-union ratio between the predicted label and the calibration label corresponding to the positive sample candidate area, and sum up the largest preset number of intersection-and-union ratios in the intersection-and-union ratios to obtain the sum result, and then round up the sum result, and use the rounded result as the number of candidate positive samples corresponding to the calibration label.

[0220] S506, determining the number of positive samples;

[0221] The electronic device can calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula, and then multiply the quantity coefficient by the number of candidate positive samples to determine the number of positive samples in the training image.

[0222]

[0223] S507, calculating target loss;

[0224] For each positive sample candidate region, the electronic device may calculate the target loss corresponding to the positive sample candidate region according to the following formula based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region.

[0225] cost = f(x)cost iou，conf +(1-f(x))dist(gt,pos box )

[0226]

[0227] cost iou，conf =iou α +conf β

[0228] Among them, center gt Used to represent the center point coordinates of the predicted label, center pos_box Used to indicate the coordinates of the center point of the calibration label; iou is the intersection-over-union ratio of the positive sample candidate area, conf is the confidence of the positive sample candidate area, α is the first preset threshold, β is the second preset threshold, and f(x) is the quantity coefficient.

[0229] S508, taking the positive sample candidate region corresponding to the target loss with the smallest number of positive samples in the target loss as the positive sample, and taking the other regions included in the training image as the negative sample;

[0230] The electronic device may use the positive sample candidate area corresponding to the target loss with the smallest number of positive samples in the target loss as the positive sample, and determine other areas in the training image except the positive sample as the negative sample.

[0231] S509, calculating the face detection loss until the initial face detection model meets the convergence condition, thereby obtaining the face detection model.

[0232] The electronic device can calculate the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample, and the difference between the predicted label and the calibrated label corresponding to the negative sample, and adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence condition, thereby obtaining the face detection model.

[0233] It can be seen that in the scheme provided in the embodiment of the present application, the electronic device can determine the positive samples and negative samples in the training image based on the difference between the predicted labels and the calibration labels corresponding to each area included in each training image, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, calculate the face detection loss based on the difference between the predicted labels and the calibration labels corresponding to the positive samples, and the difference between the predicted labels and the calibration labels corresponding to the negative samples, and then adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence conditions to obtain a face detection model. In this way, when the image to be detected is input into the face detection model, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range, so as to reduce false detection of faces in the image and improve the accuracy of face detection.

[0234] Corresponding to the training method of the above-mentioned face detection model, the embodiment of the present application further provides a face detection method. The face detection method provided by the embodiment of the present application is introduced below.

[0235] like Figure 6 As shown, a face detection method comprises:

[0236] S601, obtaining an image to be detected;

[0237] S602, inputting the image to be detected into a pre-trained face detection model to determine whether the face detection model outputs a face detection result;

[0238] The face detection model is trained according to the face detection model training method described in the above embodiment.

[0239] S603, if the face detection model does not output a face detection result, rotating the image to be detected according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected;

[0240] Wherein, the preset angle is determined based on the preset angle threshold range.

[0241] S604, returning to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state;

[0242] S605: If the face detection model outputs a face detection result, determine that the face detection result is the face detection result of the image to be detected.

[0243] It can be seen that in the scheme provided in the embodiment of the present application, the electronic device can obtain an image to be detected, input the image to be detected into a pre-trained face detection model, and determine whether the face detection model outputs a face detection result, wherein the face detection model is trained according to the training method of the face detection model described in the above embodiment; if the face detection model does not output a face detection result, the image to be detected is rotated according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected, wherein the preset angle is determined based on a preset angle threshold range, and the step of inputting the image to be detected into the pre-trained face detection model is returned until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state; if the face detection model outputs a face detection result, the face detection result is determined to be the face detection result of the image to be detected. Since the face detection model is obtained based on positive samples and negative samples training, the number of positive samples and the number of negative samples are determined based on the rotation angle of the face in the training image and the preset angle threshold range. Therefore, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. For images in which the rotation angle of the face exceeds the preset angle threshold range, the image is rotated in a preset direction and at a preset angle, so that the face in the image is rotated accordingly, so as to achieve face detection when the rotation angle of the face is within the preset angle threshold range. This can reduce false detection and missed detection of faces in images and improve the accuracy of face detection.

[0244] In step S601, the electronic device may obtain an image to be detected, wherein the image to be detected may be an image without a human face, an image with one human face, or an image with multiple human faces.

[0245] When there is a face in the image to be detected, the face may have a certain rotation angle. In one embodiment, the image to be detected has a certain rotation angle, so that the face in the image to be detected also has a certain rotation angle. In another embodiment, the image to be detected is not rotated, and the face in the image to be detected has a certain rotation angle.

[0246] For example, when a user uploads his ID card when registering an account, he inverts the direction of the ID card, so that the face in the ID card is upside down. Then, the electronic device obtains the ID card, and the face in the ID card has a rotation angle of 180 degrees.

[0247] In step S602, the electronic device may input the image to be detected into a pre-trained face detection model to determine whether the face detection model outputs a face detection result. That is, the face detection model may detect the face in the image to be detected to determine whether there is a face within a preset angle threshold range in the image to be detected.

[0248] The face detection model is pre-trained based on positive samples and negative samples included in the training image, the positive sample is a sample used to predict the face in the training image, and the negative sample is a sample not used to predict the face in the training image, and the number of positive samples and the number of negative samples are determined based on the rotation angle of the face in the training image and the preset angle threshold range. The preset angle threshold range can be the range in which the accurate face can be detected as much as possible, such as 0 to 45°, which is not specifically limited here.

[0249] That is to say, there are faces with a certain rotation angle in the training images used to train the face detection model, and the rotation angle is correlated with the preset angle threshold range. The number of positive samples and the number of negative samples are determined based on the rotation angle of the face in the training image and the preset angle threshold range. Then, by training the face detection model based on the positive samples and negative samples, the trained face detection model can detect faces with a rotation angle within the preset angle threshold range. When the rotation angle of the face exceeds the range, the detection result of the face will not be output to avoid false detection.

[0250] For example, the image to be detected obtained by the electronic device is a ID photo, and the face in the ID photo has a rotation angle of 180 degrees, that is, it is upside down. If an untrained face detection model is used to detect the face in the ID photo, since the face is upside down, the part from the eyes to the head may be detected, resulting in false detection of the face.

[0251] The pre-trained face detection model can suppress the detection of faces that cannot be accurately detected due to a large rotation angle, that is, it will not detect faces with a large rotation angle. Therefore, the electronic device can input the ID photo into the pre-trained face detection model, and the preset angle threshold range of the face detection model is [0, 90°), so the face detection result of the face with a rotation angle of 180 degrees will not be output.

[0252] In step S603, if the face detection model does not output a face detection result, the electronic device may rotate the image to be detected according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected.

[0253] Specifically, if the face detection model does not output a face detection result, there are two situations: one is that there is no face in the image to be detected, and the other is that there is a face with a rotation angle that is too large. In order to avoid missing the detection of faces with a rotation angle that is too large, the electronic device can rotate the image to be detected, so that the face in the image to be detected will also be rotated accordingly, and the rotation angle of the face will change, and it may be within the detectable range. Therefore, the electronic device can use the rotated image to be detected as the image to be detected, input it into the face detection model to perform face detection on the image to be detected, and determine whether the face detection model outputs a face detection result.

[0254] The preset angle is determined based on a preset angle threshold range. Generally, if the preset angle threshold range is relatively large, then the preset angle is relatively large, and if the preset angle threshold range is relatively small, then the preset angle is relatively small. The preset direction can be clockwise or counterclockwise, and is not specifically limited here.

[0255] For example, if an upside-down ID photo is input into the face detection model, and the face detection model does not output a face detection result, the electronic device can rotate the ID photo 90° counterclockwise and input the rotated ID photo into the face detection model, so that the face detection model performs face detection on the rotated ID photo.

[0256] After the electronic device reacquires the image to be detected, in step S604, it can continue to input the image to be detected into a pre-trained face detection model to perform face detection on the image to be detected until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state.

[0257] Since the image to be detected will return to its initial state after being rotated a certain number of times, or has been rotated 360°, even if the electronic device continues to rotate the image to be detected, the detection result of the image to be detected by the face detection model will not change. Therefore, if the face detection model has not output a face detection result after the electronic device rotates the image to be detected multiple times, it can be determined whether the image to be detected has been rotated to its initial state. If it has been rotated to its initial state, the detection of the image to be detected is terminated, and the face detection result output by the face detection model is empty.

[0258] Of course, when the image to be detected rotates to the position where it was last rotated to the initial state, if one more rotation can reach the initial state or the rotation exceeds 360°, then the rotation of the image to be detected can be stopped when the image to be detected rotates to the position where it was last rotated to the initial state.

[0259] In one embodiment, the electronic device can calculate the number of rotations according to the preset rotation angle as the target number of rotations, so as to complete the detection of the image to be detected when the number of rotations reaches the target number of rotations. In another embodiment, the electronic device can calculate the total angle of rotation according to the rotation angle and the number of rotations, and then complete the detection of the image to be detected when the total angle of rotation reaches 360°.

[0260] For example, if the preset angle is 90°, and the electronic device rotates the image to be detected according to the preset angle, whether it is rotated counterclockwise or clockwise, the image to be detected will be rotated to the initial state after 4 rotations. In order to reduce the number of detections and improve detection efficiency, the electronic device can rotate the image to be detected 3 times, that is, rotate it to the position of the previous rotation to the initial state. In this way, when the number of rotations reaches 3 times and the face detection model does not output a face detection result, the detection of the image to be detected is completed.

[0261] If the rotation angle of the face in the image to be detected is 180 degrees, then the electronic device rotates the image to be detected twice in a counterclockwise direction and inputs it into the face detection model, and the face detection model can output the face detection result.

[0262] In step S605, if the face detection model outputs a face detection result, the face detection result is determined to be the face detection result of the image to be detected. In face detection, if the electronic device uses a certain strategy to search the image to determine whether there is a face in the image, the position, size and posture of the face can be obtained.

[0263] In one embodiment, if there is a face in the image to be detected, the electronic device can determine the position and size of the face in the image to be detected, and represent the face position and size with a rectangular frame represented by a 4-dimensional array. Then, if the face detection model detects a face, it can output a rectangular frame corresponding to the face, and then use the rectangular frame as the face detection result of the image to be detected.

[0264] For example, if there is face A in an upside-down ID photo, when the electronic device rotates the ID photo 180 degrees and inputs it into the face detection model, the face detection model can detect face A, and then frame face A with a rectangular frame a to indicate the size and position of the face. In this way, the electronic device can determine that the rectangular frame a is the face detection result of the ID photo.

[0265] In the scheme of the present embodiment, since the face detection model is obtained based on positive sample and negative sample training, the number of positive samples and the number of negative samples are determined based on the rotation angle of the face in the training image and the preset angle threshold range. Therefore, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. For images in which the rotation angle of the face exceeds the preset angle threshold range, the face is rotated according to a preset direction and a preset angle, so that the face in the image is rotated accordingly, so as to achieve face detection when the rotation angle of the face is within the preset angle threshold range. This can reduce false detection and missed detection of faces in images and improve the accuracy of face detection.

[0266] In addition, the face detection method provided in the present application has the same accuracy and speed as the general face detection model for other scenarios in which face detection is performed, and can solve face detection problems in all scenarios with only one face detection model.

[0267] As an implementation manner of an embodiment of the present application, the above-mentioned preset angle enables the rotation angle of the face in the image to be detected to fall within the preset angle threshold range at least once during the rotation process of the image to be detected.

[0268] Since the electronic device rotates the image to be detected in order to make the face in the image to be detected after the rotation, that is, the rotation angle of the face is within the detectable range, then the preset angle can make the rotation angle of the face in the image to be detected fall within the preset angle threshold range at least once during the rotation of the image to be detected.

[0269] For example, the preset angle threshold range is [0, 45°), and the rotation angle of the face in the image to be detected is 90°. Then, when the preset angle is 90°, the rotation angle of the face in the image to be detected when the image to be detected is rotated for the third time is within the range of [0, 45°). When the preset angle is 60°, the rotation angle of the face in the image to be detected when the image to be detected is rotated for the fifth time is within the range of [0, 45°].

[0270] Figure 7 A specific flow chart of the face detection model training method provided in the embodiment of the present application. Figure 7 The face detection model training method provided in the embodiment of the present application is introduced by way of example. Figure 7 As shown, the face detection model training method provided in the embodiment of the present application may include the following steps:

[0271] S701, inputting the image to be detected into the face detection model;

[0272] When the electronic device obtains the image to be detected, it can input the image to be detected into the face detection model. The face detection model can detect the face whose rotation angle is within the preset angle threshold range. For example, the image to be detected is an ID card uploaded by the user, and the electronic device inputs the ID card into the face detection model so that the face detection model detects the face in the ID card.

[0273] S702, whether a face frame is detected; if yes, execute step S703, if not, execute step S704;

[0274] If the face detection model detects a face, it will output the face detection result, that is, the face frame. If the face detection model does not detect a face, it will not output the face detection result, that is, the face frame. At this time, the electronic device can rotate the image to be detected in a certain direction and angle to input the rotated face into the face detection model and perform face detection again.

[0275] S703, outputting the face detection result;

[0276] When the face detection model detects a face, the face detection result, that is, the face frame, can be output. The electronic device can obtain the face frame to use the face frame as the face detection result of the image to be detected.

[0277] S704: Input the image to be detected rotated 90° counterclockwise.

[0278] When the face detection model does not output a face detection result, the electronic device can rotate the image to be detected in a certain direction and angle. For example, the image to be detected can be rotated 90° counterclockwise until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state.

[0279] For example, the ID card uploaded by the user is upside down, and the face in the ID card is also upside down, that is, rotated 180°. The preset angle threshold range of the face detection model is [0, 60°), so the face detection model will detect the ID card at this time and will not output the face detection result. The electronic device can rotate the ID card counterclockwise by 90°, and the face rotation angle is 270°. The face detection model will detect the ID card and still will not output the face detection result. The electronic device rotates the ID card counterclockwise by another 90°, and the face rotation angle is 306°, that is, 0°. The face detection model can detect the face and output the face detection result, that is, output the face frame.

[0280] It can be seen that in the solution provided in the embodiment of the present application, the electronic device can obtain an image to be detected, and input the image to be detected into a face detection model. When the rotation angle of the face is within the preset angle threshold range, the face prediction model can output a face detection result. When the rotation angle of the face exceeds the preset angle threshold range, the face detection model suppresses the output of the face detection result. Therefore, for an image in which the rotation angle of the face exceeds the preset angle threshold range, it can be rotated in a preset direction and at a preset angle, so that the face in the image is rotated accordingly, so as to achieve face detection when the rotation angle of the face is within the preset angle threshold range. This can reduce false detection and missed detection of faces in images and improve the accuracy of face detection.

[0281] Corresponding to the above-mentioned training method of the face detection model, the embodiment of the present application further provides a training device for the face detection model. The following is an introduction to the training device for the face detection model provided in the embodiment of the present application.

[0282] like Figure 8 As shown, a training device for a face detection model, the device comprising:

[0283] A first acquisition module 810 is used to acquire a plurality of training images and a calibration label corresponding to each training image;

[0284] A second acquisition module 820 is used to input each training image into an initial face detection model to obtain a prediction label corresponding to each region included in each training image;

[0285] A positive and negative sample determination module 830 is used to determine, for each training image, the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0286] A face detection loss calculation module 840, configured to calculate the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample, and the difference between the predicted label and the calibrated label corresponding to the negative sample;

[0287] The third acquisition module 850 is used to adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence condition, thereby obtaining the face detection model.

[0288] Among them, the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

[0289] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain multiple training images and the calibration labels corresponding to each training image; input each training image into the initial face detection model to obtain the predicted labels corresponding to each area included in each training image; for each training image, according to the difference between the predicted labels and the calibration labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, determine the positive sample and the negative sample in the training image; based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample, calculate the face detection loss; based on the face detection loss, adjust the parameters of the initial face detection model until the initial face detection model meets the convergence condition, and obtain the face detection model. Among them, the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range. Since the face detection model is obtained based on positive and negative sample training, the number of positive samples and negative samples is determined according to the difference between the predicted label and the calibration label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range. For the input image to be detected, when the rotation angle of the face in the image to be detected is within the preset angle threshold range, the face detection model outputs the face detection result, and when the rotation angle of the face in the image to be detected is not within the preset angle threshold range, the face detection model does not output the face detection result. Therefore, when using the face detection model for face detection, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. This can reduce false detection and missed detection of faces in the image and improve the accuracy of face detection.

[0290] As an implementation of the embodiment of the present application, the positive and negative sample determination module 830 may include:

[0291] A positive sample candidate region determining unit, configured to determine a positive sample candidate region from each region included in the training image based on the center point position of the calibration label corresponding to the training image;

[0292] A positive sample quantity determination unit, used to determine the positive sample quantity of the positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range;

[0293] A target loss calculation unit is used to calculate, for each positive sample candidate region, a target loss corresponding to the positive sample candidate region based on a difference between a predicted label and a calibrated label corresponding to the positive sample candidate region, wherein the target loss includes a position prior loss, a sample regression loss, and a sample classification loss;

[0294] A positive sample determining unit, configured to determine the number of positive sample candidate regions from the positive sample candidate regions as positive samples based on the target loss corresponding to each positive sample candidate region;

[0295] The negative sample determination unit is used to determine other areas in the training image except the positive samples as negative samples.

[0296] As an implementation of the embodiment of the present application, the above-mentioned positive sample quantity determination unit may include:

[0297] An intersection-over-union ratio determination subunit is used to calculate, for each positive sample candidate region, an intersection-over-union ratio between a predicted label and a calibrated label corresponding to the positive sample candidate region;

[0298] A summation result acquisition subunit is used to sum a maximum preset number of intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result;

[0299] A candidate positive sample quantity acquisition subunit is used to round the sum result as the candidate positive sample quantity corresponding to the calibration label;

[0300] The positive sample quantity determination subunit is used to determine the positive sample quantity of the positive samples in the training image based on the candidate positive sample quantity and the relationship between the rotation angle of the face in the training image and the preset angle threshold range.

[0301] As an implementation of an embodiment of the present application, the above-mentioned positive sample quantity determination subunit may include:

[0302] A quantity coefficient calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the human face in the training image and the preset angle threshold range and the preset coefficient determination rule, wherein the preset coefficient determination rule is such that: when the rotation angle of the human face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the human face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the human face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range and greater than the preset angle threshold range, and the first preset value is greater than the second preset value and greater than 0;

[0303] The first determining subunit is used to determine the product of the quantity coefficient and the number of candidate positive samples as the number of positive samples in the training image.

[0304] As an implementation of an embodiment of the present application, the above-mentioned quantity coefficient calculation subunit may include:

[0305] A first calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula;

[0306]

[0307] Among them, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range.

[0308] As an implementation of an embodiment of the present application, the target loss calculation unit may include:

[0309] A confidence acquisition subunit, used to obtain the confidence corresponding to the positive sample candidate region;

[0310] The prior center distance calculation subunit is used to calculate the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label according to the following formula;

[0311]

[0312] Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label;

[0313] A loss calculation subunit, configured to calculate the loss of the positive sample candidate region according to the following formula based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, wherein the loss includes sample regression loss and sample classification loss;

[0314] cost iou，conf =iou α +conf β

[0315] Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold;

[0316] The target loss calculation subunit is used to calculate the target loss corresponding to the positive sample candidate region based on the prior center distance and the loss according to the following formula:

[0317] cost = f(x)cost iou，conf +(1-f(x))dist(gt,pos box )

[0318] Wherein, f(x) is the quantitative coefficient.

[0319] As an implementation of an embodiment of the present application, the positive sample determination unit may include:

[0320] The positive sample determination subunit is used to take the positive sample candidate region corresponding to the smallest number of positive sample target losses in the target losses as the positive sample based on the target loss corresponding to each positive sample candidate region.

[0321] Corresponding to the above-mentioned face detection method, the embodiment of the present application further provides a face detection device. The face detection device provided by the embodiment of the present application is introduced below.

[0322] like Fig. 9 As shown, a face detection device, the device comprising:

[0323] The image acquisition module 910 is used to acquire the image to be detected;

[0324] A detection result judgment module 920 is used to input the image to be detected into a pre-trained face detection model to determine whether the face detection model outputs a face detection result;

[0325] The face detection model is trained according to the face detection model training method described in the above embodiment.

[0326] The image to be detected rotation module 930 is configured to rotate the image to be detected according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected if the face detection model does not output a face detection result, wherein the preset angle is determined based on the preset angle threshold range;

[0327] A detection result acquisition module 940 is used to return to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state;

[0328] The face detection result determination module 950 is used to determine that the face detection result is the face detection result of the image to be detected if the face detection model outputs a face detection result.

[0329] It can be seen that in the scheme provided in the embodiment of the present application, the electronic device can obtain an image to be detected, input the image to be detected into a pre-trained face detection model, and determine whether the face detection model outputs a face detection result, wherein the face detection model is trained according to the training method of the face detection model described in the above embodiment; if the face detection model does not output a face detection result, the image to be detected is rotated according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected, wherein the preset angle is determined based on a preset angle threshold range, and the step of inputting the image to be detected into the pre-trained face detection model is returned until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state; if the face detection model outputs a face detection result, the face detection result is determined to be the face detection result of the image to be detected. Since the face detection model is obtained based on positive samples and negative samples training, the number of positive samples and the number of negative samples are determined based on the rotation angle of the face in the training image and the preset angle threshold range. Therefore, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. For images in which the rotation angle of the face exceeds the preset angle threshold range, the image is rotated in a preset direction and at a preset angle, so that the face in the image is rotated accordingly, so as to achieve face detection when the rotation angle of the face is within the preset angle threshold range. This can reduce false detection and missed detection of faces in images and improve the accuracy of face detection.

[0330] As an implementation manner of an embodiment of the present application, the preset angle enables the rotation angle of the face in the image to be detected to fall within the preset angle threshold range at least once during the rotation of the image to be detected.

[0331] The present application also provides an electronic device, such as Fig.10 As shown, including:

[0332] Memory 1001, used for storing computer programs;

[0333] The processor 1002 is used to implement the face detection model training method or face detection method described in any of the above embodiments when executing the program stored in the memory 1001.

[0334] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1002, the communication interface, and the memory 1001 communicate with each other via the communication bus.

[0335] It can be seen that in the scheme provided in the embodiment of the present application, the electronic device can obtain multiple training images and the calibration labels corresponding to each training image; input each training image into the initial face detection model to obtain the predicted labels corresponding to each area included in each training image; for each training image, according to the difference between the predicted labels and the calibration labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, determine the positive sample and the negative sample in the training image; based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample, calculate the face detection loss; based on the face detection loss, adjust the parameters of the initial face detection model until the initial face detection model meets the convergence condition, and obtain the face detection model. Among them, the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range. Since the face detection model is obtained based on positive and negative sample training, the number of positive samples and negative samples is determined according to the difference between the predicted label and the calibration label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range. For the input image to be detected, when the rotation angle of the face in the image to be detected is within the preset angle threshold range, the face detection model outputs the face detection result, and when the rotation angle of the face in the image to be detected is not within the preset angle threshold range, the face detection model does not output the face detection result. Therefore, when using the face detection model for face detection, the face detection result can be output when the rotation angle of the face is within the preset angle threshold range, and the output of the face detection result can be suppressed when the rotation angle of the face exceeds the preset angle threshold range. This can reduce false detection and missed detection of faces in the image and improve the accuracy of face detection.

[0336] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0337] The communication interface is used for communication between the above electronic device and other devices.

[0338] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0339] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0340] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned face detection model training methods or face detection methods.

[0341] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any face detection model training method or face detection method in the above-mentioned embodiments.

[0342] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0343] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0344] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0345] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A training method for a face detection model, characterized in that: The method comprises: Obtain multiple training images and a calibration label corresponding to each training image; Inputting each training image into an initial face detection model to obtain a prediction label corresponding to each region included in each training image; For each training image, determine the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range; Calculating the face detection loss based on the difference between the predicted label and the calibrated label corresponding to the positive sample and the difference between the predicted label and the calibrated label corresponding to the negative sample; Based on the face detection loss, the parameters of the initial face detection model are adjusted until the initial face detection model meets the convergence condition, so as to obtain the face detection model, wherein the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

2. The method according to claim 1, characterized in that The step of determining positive samples and negative samples in the training image according to the difference between the predicted labels and the calibrated labels corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range includes: Based on the center point position of the calibration label corresponding to the training image, determining a positive sample candidate region from each region included in the training image; Determine the number of positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region and the relationship between the rotation angle of the face in the training image and the preset angle threshold range; For each positive sample candidate region, based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region, the target loss corresponding to the positive sample candidate region is calculated, wherein the target loss includes position prior loss, sample regression loss and sample classification loss; Based on the target loss corresponding to each positive sample candidate region, determining the number of positive sample candidate regions from the positive sample candidate regions as positive samples; Other regions in the training image except the positive samples are determined as negative samples.

3. The method according to claim 2, characterized in that The step of determining the number of positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region and the relationship between the rotation angle of the face in the training image and the preset angle threshold range includes: For each positive sample candidate region, calculate the intersection-over-union ratio between the predicted label and the calibrated label corresponding to the positive sample candidate region; Summing a maximum preset number of intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result; The sum result is rounded to an integer to obtain the number of candidate positive samples corresponding to the calibration label; Based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range, the number of positive samples in the training image is determined.

4. The method according to claim 3, characterized in that The step of determining the number of positive samples of the positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and the preset angle threshold range comprises: Based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule, the quantity coefficient corresponding to the training image is calculated, wherein the preset coefficient determination rule is such that: when the rotation angle of the face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range and greater than the preset angle threshold range, and the first preset value is greater than the second preset value and greater than 0; The product of the quantity coefficient and the number of candidate positive samples is determined as the number of positive samples in the training image.

5. The method according to claim 4, characterized in that The step of calculating the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule comprises: Based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula, the quantity coefficient corresponding to the training image is calculated; Among them, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range.

6. The method according to claim 2, characterized in that The step of calculating the target loss corresponding to the positive sample candidate region based on the difference between the predicted label and the calibrated label corresponding to the positive sample candidate region includes: Obtain the confidence level corresponding to the positive sample candidate region; According to the center point position coordinates of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label, the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position of the calibration label is calculated according to the following formula; Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label; Based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, the loss of the positive sample candidate region is calculated according to the following formula, wherein the loss includes sample regression loss and sample classification loss; cost iou,conf =iou α +conf β Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold; Based on the prior center distance and the loss, the target loss corresponding to the positive sample candidate region is calculated according to the following formula: cost=f(x)cost iou，conf +(1-f(x))dist(gt,pos box ) Wherein, f(x) is the quantitative coefficient.

7. The method according to claim 2, characterized in that The step of determining the number of positive sample candidate regions as positive samples from the positive sample candidate regions based on the target loss corresponding to each positive sample candidate region, comprises: Based on the target loss corresponding to each positive sample candidate region, the positive sample candidate region corresponding to the target loss with the smallest number of positive samples in the target loss is taken as a positive sample.

8. A face detection method, characterized in that: The method comprises: Acquire the image to be detected; Inputting the image to be detected into a pre-trained face detection model, and determining whether the face detection model outputs a face detection result, wherein the face detection model is pre-trained according to the method according to any one of claims 1 to 7; If the face detection model does not output a face detection result, rotating the image to be detected according to a preset direction and a preset angle to obtain a rotated image to be detected as the image to be detected, wherein the preset angle is determined based on the preset angle threshold range; Returning to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state; If the face detection model outputs a face detection result, it is determined that the face detection result is the face detection result of the image to be detected.

9. The method according to claim 8, characterized in that The preset angle ensures that, during the rotation of the image to be detected, the rotation angle of the face in the image to be detected falls within the preset angle threshold range at least once.

10. A training device for a face detection model, characterized in that: The device comprises: A first acquisition module is used to acquire a plurality of training images and a calibration label corresponding to each training image; A second acquisition module is used to input each training image into an initial face detection model to obtain a prediction label corresponding to each area included in each training image; A positive and negative sample determination module is used to determine, for each training image, the positive sample and the negative sample in the training image according to the difference between the predicted label and the calibrated label corresponding to each area included in the training image and the relationship between the rotation angle of the face in the training image and the preset angle threshold range; A face detection loss calculation module, used to calculate the face detection loss based on the difference between the predicted label and the calibration label corresponding to the positive sample, and the difference between the predicted label and the calibration label corresponding to the negative sample; A third acquisition module is used to adjust the parameters of the initial face detection model based on the face detection loss until the initial face detection model meets the convergence condition, thereby obtaining the face detection model, wherein the face detection model is used to perform face detection on the input image to be detected, and output the face detection result when the rotation angle of the face in the image to be detected is within the preset angle threshold range, and do not output the face detection result when the rotation angle of the face in the image to be detected is not within the preset angle threshold range.

11. The device according to claim 10, characterized in that The positive and negative sample determination module comprises: A positive sample candidate region determining unit, configured to determine a positive sample candidate region from each region included in the training image based on the center point position of the calibration label corresponding to the training image; A positive sample quantity determination unit, used to determine the positive sample quantity of the positive samples in the training image according to the difference between the predicted label and the calibrated label corresponding to each positive sample candidate region, and the relationship between the rotation angle of the face in the training image and the preset angle threshold range; A target loss calculation unit is used to calculate, for each positive sample candidate region, a target loss corresponding to the positive sample candidate region based on a difference between a predicted label and a calibrated label corresponding to the positive sample candidate region, wherein the target loss includes a position prior loss, a sample regression loss, and a sample classification loss; A positive sample determining unit, configured to determine the number of positive sample candidate regions from the positive sample candidate regions as positive samples based on the target loss corresponding to each positive sample candidate region; A negative sample determination unit, used for determining other regions in the training image except the positive samples as negative samples; The positive sample quantity determination unit comprises: An intersection-over-union ratio determination subunit is used to calculate, for each positive sample candidate region, an intersection-over-union ratio between a predicted label and a calibrated label corresponding to the positive sample candidate region; A summation result acquisition subunit is used to sum a maximum preset number of intersection-and-union ratios among the intersection-and-union ratios to obtain a summation result; A candidate positive sample quantity acquisition subunit is used to round the sum result as the candidate positive sample quantity corresponding to the calibration label; A positive sample number determination subunit, used to determine the number of positive samples in the training image based on the number of candidate positive samples and the relationship between the rotation angle of the face in the training image and a preset angle threshold range; The positive sample quantity determination subunit includes: A quantity coefficient calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the human face in the training image and the preset angle threshold range and the preset coefficient determination rule, wherein the preset coefficient determination rule is such that: when the rotation angle of the human face is within the preset angle threshold range, the quantity coefficient is a first preset value; when the rotation angle of the human face is within the first preset range, the quantity coefficient is a second preset value; when the rotation angle of the human face is within the second preset range, the quantity coefficient is 0, the second preset range is greater than the first preset range and greater than the preset angle threshold range, and the first preset value is greater than the second preset value and greater than 0; A first determining subunit, configured to determine the number of positive samples of the training image as the number of positive samples by multiplying the number coefficient by the number of candidate positive samples; The quantity coefficient calculation subunit includes: A first calculation subunit is used to calculate the quantity coefficient corresponding to the training image based on the relationship between the rotation angle of the face in the training image and the preset angle threshold range and the preset coefficient determination rule shown in the following formula; Wherein, x is the rotation angle of the face, [0, θ) is the preset angle threshold range, [θ, 90) is the first preset range, and [90, 180] is the second preset range; The target loss calculation unit comprises: A confidence acquisition subunit, used to obtain the confidence corresponding to the positive sample candidate region; The prior center distance calculation subunit is used to calculate the prior center distance between the center point position of the predicted label corresponding to the positive sample candidate area and the center point position coordinates of the calibration label according to the following formula; Among them, center gt Used to represent the center point position coordinates of the predicted label, center pos_box Used to indicate the center point position coordinates of the calibration label; A loss calculation subunit, configured to calculate the loss of the positive sample candidate region according to the following formula based on the confidence and intersection-over-union ratio corresponding to the positive sample candidate region, wherein the loss includes sample regression loss and sample classification loss; cost iou，conf =iou α +conf β Wherein, iou is the intersection-over-union ratio, conf is the confidence level, α is the first preset threshold, and β is the second preset threshold; The target loss calculation subunit is used to calculate the target loss corresponding to the positive sample candidate region based on the prior center distance and the loss according to the following formula: cost=f(x)cost iou，conf +(1-f(x))dist(gt,pos box ) Wherein, f(x) is the quantitative coefficient; The positive sample determination unit comprises: The positive sample determination subunit is used to take the positive sample candidate region corresponding to the smallest number of positive sample target losses in the target losses as the positive sample based on the target loss corresponding to each positive sample candidate region.

12. A face detection device, characterized in that: The device comprises: An image acquisition module to be detected is used to acquire the image to be detected; A detection result judgment module, used for inputting the image to be detected into a pre-trained face detection model to determine whether the face detection model outputs a face detection result, wherein the face detection model is pre-trained according to the method according to any one of claims 1 to 7; A detection image rotation module, if the face detection model does not output a face detection result, rotates the detection image according to a preset direction and a preset angle to obtain a rotated detection image as the detection image, wherein the preset angle is determined based on the preset angle threshold range; A detection result acquisition module, used for returning to the step of inputting the image to be detected into a pre-trained face detection model until the face detection model outputs a face detection result, or the image to be detected is rotated to an initial state; The face detection result determination module is used to determine that the face detection result is the face detection result of the image to be detected if the face detection model outputs the face detection result.

13. The device according to claim 12, characterized in that The preset angle ensures that, during the rotation of the image to be detected, the rotation angle of the face in the image to be detected falls within the preset angle threshold range at least once.

14. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-9 when executing a program stored in a memory.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images