Face detection methods, devices, electronic equipment and storage media

Abstract

This invention provides a face detection method, apparatus, electronic device, and storage medium. The method includes: acquiring an image to be detected, the image including an occluded face; inputting the image to be detected into a pre-trained face detection model, and outputting a face detection result through the pre-trained face detection model; wherein the pre-trained face detection model incorporates auxiliary detection based on local facial features, and the pre-trained face detection model is trained using a local facial dataset. By using a face detection model with local facial features as auxiliary detection, face detection is performed on the image to be detected, which is equivalent to performing prior detection on the local facial features before detecting the face, thus improving the detection accuracy of occluded faces.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a face detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the deepening of artificial intelligence research, image recognition products are becoming more and more abundant, such as access control, monitoring, intelligent traffic and other products based on face recognition. Face recognition technology includes face detection and extraction, face image preprocessing, face feature extraction and face feature comparison and recognition. Due to the influence of the camera shooting angle or the person actively shielding the face, the face image may be shielded, for example, when the person wears a duckbill cap, the camera set above the site is easy to shoot a part of the face image shielded by the duckbill cap, and for example, when the user wears a mask or protective glasses, the face image captured by the camera is partially shielded. Due to the shielding of the face image, the complete face feature cannot be extracted in the face detection process, thereby reducing the accuracy of face detection. For the shielded face, one solution is to detect the highly shielded face as a face label, which increases the pressure on the model and is easy to cause false detection. Another solution is to distinguish between normal faces and highly shielded faces for two-class face detection. This solution is difficult to give a clear boundary between normal and highly shielded faces, and is also easy to cause false detection. Therefore, the detection accuracy of the shielded face in the existing face recognition is not high. SUMMARY

[0003] The embodiment of the present application provides a face detection method, which can improve the accuracy of face detection between persons.

[0004] In a first aspect, the embodiment of the present application provides a face detection method, which comprises:

[0005] Obtaining a to-be-detected image, the to-be-detected image comprising a shielded face;

[0006] Inputting the to-be-detected image into a pre-trained face detection model, and outputting a face detection result through the pre-trained face detection model;

[0007] Wherein, the pre-trained face detection model is fitted with auxiliary detection of face local features, and the pre-trained face detection model is obtained through auxiliary training of a face local data set.

[0008] Optionally, the untrained face detection model comprises a fitting network and a first branch network and a second branch network, the fitting network is connected with the first branch network and the second branch network, wherein the first branch network is used to assist the training of the fitting network, so that the fitting network fits the auxiliary detection of the local features of the face, and the trained face detection model comprises the fitting network and the second branch network.

[0009] Optionally, the training of the untrained face detection model comprises:

[0010] The fitting network and the first branch network are trained by using a face local data set until the fitting network and the first branch network converge, so as to obtain the converged fitting network and the converged first branch network.

[0011] The parameters of the converged fitting network are fixed, and the converged fitting network and the second branch network are trained by using a face data set, so as to obtain the converged second branch network, wherein the face sample images in the face data set comprise face local images.

[0012] The trained face detection model is obtained based on the converged fitting network and the converged second branch network.

[0013] Optionally, the obtaining of the trained face detection model based on the converged fitting network and the converged second branch network comprises:

[0014] The trained face detection model is obtained by cutting the converged first branch network, and the trained face detection model only comprises the converged fitting network and the converged second branch network.

[0015] Optionally, the training of the fitting network and the first branch network by using the face local data set comprises:

[0016] The face local data set is obtained, and the face local data set comprises face local image samples and corresponding face local annotation data.

[0017] The face local image samples are input into the fitting network and the first branch network for calculation, so as to obtain a first calculation result, and a first loss function between the first calculation result and the face local annotation data is calculated.

[0018] According to the first loss function, the parameters of the fitting network and the first branch network are adjusted, and iteration is performed until the fitting network and the first branch network converge.

[0019] Optionally, the face local annotation data comprises face local bounding box annotation data and / or face local key point annotation data, the first calculation result comprises face local bounding box data and / or face local key point data, the first loss function is a loss function between the face local bounding box data and the face local bounding box annotation data, or the first loss function is a loss function between the face local key point data and the face local key point annotation data, or the first loss function is a total loss function of a loss function between the face local bounding box data and the face local bounding box annotation data and a loss function between the face local key point data and the face local key point annotation data.

[0020] Optionally, the parameters of the converged fitting network are fixed, and the converged fitting network and the second branch network are trained by using the face data set, comprising:

[0021] obtaining a face data set, wherein the face data set comprises face image samples and corresponding face annotation data;

[0022] fixing the parameters of the converged fitting network;

[0023] inputting the face image samples into the converged fitting network and the second branch network to obtain a second calculation result, and calculating a second loss function between the second calculation result and the face annotation data;

[0024] performing back propagation according to the second loss function to adjust the parameters of the second branch network, and iteratively converging the second branch network.

[0025] Optionally, the face local feature is an eye feature, and the pre-trained face detection model is obtained by auxiliary training using an eye data set.

[0026] In a second aspect, the embodiments of the present application further provide a face detection device, comprising:

[0027] a obtaining module, configured to obtain a to-be-detected image, wherein the to-be-detected image comprises an occluded face;

[0028] a detection module, configured to input the to-be-detected image into a pre-trained face detection model, and output a face detection result by using the pre-trained face detection model;

[0029] Optionally, the pre-trained face detection model is fitted with auxiliary detection of a face local feature, and the pre-trained face detection model is obtained by auxiliary training using a face local data set.

[0030] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the face detection method provided in embodiments of the present invention.

[0031] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the face detection method provided in the embodiments of the present invention.

[0032] In this embodiment of the invention, an image to be detected is acquired, the image including an occluded face; the image is input into a pre-trained face detection model, and a face detection result is obtained by outputting the pre-trained face detection model; wherein, the pre-trained face detection model incorporates auxiliary detection based on local facial features, and the pre-trained face detection model is obtained through auxiliary training using a local facial dataset. By using a face detection model with local facial features as auxiliary detection, face detection is performed on the image to be detected, which is equivalent to performing prior detection on the local facial features before detecting the face, thus improving the detection accuracy of occluded faces. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of a face detection method provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of an untrained face detection model provided in an embodiment of the present invention;

[0036] Figure 3 This is a flowchart of a face detection model training method provided in an embodiment of the present invention;

[0037] Figure 4 This is a flowchart of a fitting network and a first branch network training method provided in an embodiment of the present invention;

[0038] Figure 5 This is a flowchart of a convergent fitting network and a second branch network training method provided in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the structure of a face detection device provided in an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram of another face detection device provided in an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram of the structure of a first training module provided in an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram of the structure of a second training module provided in an embodiment of the present invention;

[0043] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 , Figure 1 This is a flowchart of a face detection method provided in an embodiment of the present invention, such as... Figure 1 As shown, it includes the following steps:

[0046] 101. Obtain the image to be detected.

[0047] In this embodiment of the invention, the image to be detected may include both occluded and unoccluded faces. The image to be detected may be captured in real-time by a camera or uploaded by a user.

[0048] The obscured face mentioned above can be a heavily obscured face, such as a face covered by a mask, helmet, or face mask.

[0049] 102. Input the image to be detected into the pre-trained face detection model, and obtain the face detection result through the output of the pre-trained face detection model.

[0050] In this embodiment of the invention, the pre-trained face detection model is fitted with auxiliary detection of local facial features, and the pre-trained face detection model is obtained through auxiliary training using a local facial dataset.

[0051] The aforementioned facial features can be areas of the face that are not easily obscured, such as the eye area, ear area, eyebrow area, and mouth area. Correspondingly, the aforementioned facial features can be one or more of the following: eye features, ear features, eyebrow features, and mouth features.

[0052] Optionally, since the human eye region has richer feature information, embodiments of the present invention may preferentially use the human eye region as a local feature of the face.

[0053] In some possible embodiments, the pre-trained face detection model can also fit multiple local facial features, such as eye features, ear features, and eyebrow features, as auxiliary detection features to further improve the accuracy of face detection.

[0054] In training a face detection model, the first step is to use a local face dataset for initial training. This allows the model to learn local facial features and fit these features into the model. Then, a second training step is performed using the same dataset to enable the model to learn face detection and output the corresponding facial information.

[0055] The face detection results mentioned above can be face bounding boxes and / or facial landmarks, which can be determined according to the actual needs of the scenario.

[0056] In embodiments of the present invention, such as Figure 2 As shown, the untrained face detection model includes a fitting network, a first branch network, and a second branch network. The fitting network is connected to both the first and second branch networks. The first branch network is used to assist in the training of the fitting network, enabling it to fit the auxiliary detection of local facial features. The trained face detection model includes both the fitting network and the second branch network. The fitting network, the first branch network, and the second branch network can be convolutional neural networks.

[0057] Specifically, for training the face detection model, please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart of a face detection model training method provided in an embodiment of the present invention, such as... Figure 3 As shown, it includes the following steps:

[0058] 301. Train the fitting network and the first branch network using a local face dataset until they converge, thus obtaining a converged fitting network and a converged first branch network.

[0059] In this embodiment of the invention, the aforementioned local face dataset can be an eye dataset, an ear dataset, an eyebrow dataset, a mouth dataset, etc. The local face dataset can be selected based on the specific application scenario of the face detection model. Specific application scenarios for the face detection model can be scenarios with face occlusion factors, such as scenarios involving wearing masks, helmets, face masks, or eye protection (glasses, goggles). For example, when monitoring scenarios involving wearing masks, helmets, or face masks, at least one dataset such as an eye dataset, ear dataset, or eyebrow dataset can be used to train an untrained face detection model. When monitoring scenarios involving eye protection, an untrained face detection model using a mouth dataset can be used for training.

[0060] Optionally, the aforementioned local face dataset includes local face annotation data, which includes local face bounding box annotation data and / or local face keypoint annotation data. The local face bounding box annotation data corresponds to local face detection, and the local face keypoint annotation data corresponds to local face keypoint detection.

[0061] Optionally, if the fitting network and the first branch network are trained using facial bounding box annotation data, the calculated result is facial bounding box information, such as eye boxes, ear boxes, eyebrow boxes, and mouth boxes. Of course, facial bounding box information can also be understood as facial local locations. The facial bounding box information can be expressed as (x, y, h, w), where x and y are the coordinates of the center point of the facial bounding box, h is the height of the facial bounding box, and w is the width of the facial bounding box. If the fitting network and the first branch network are trained using facial keypoint annotation data, the calculated result is facial keypoint information, such as eye keypoints, ear keypoints, eyebrow keypoints, and mouth keypoints corresponding to the facial bounding box type. Of course, facial keypoint information can also be understood as the location of facial keypoints. The aforementioned eye keypoints can be keypoints on the eye contour, ear keypoints can be keypoints on the ear contour, eyebrow keypoints can be keypoints on the eyebrow contour, and mouth keypoints can be keypoints on the mouth contour. If the fitted network and the first branch network are trained using face bounding box annotation data and face key point annotation data, the calculation results will include both face bounding box information and face key point information.

[0062] Specifically, for training the fitted network and the first branch network, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a flowchart of a fitting network and a first branch network training method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, it includes the following steps:

[0063] 401. Obtain a partial face dataset.

[0064] 402. Input the local face image samples into the fitting network and the first branch network for calculation to obtain the first calculation result, and calculate the first loss function between the first calculation result and the local face annotation data.

[0065] 403. Perform backpropagation based on the first loss function to adjust the parameters of the fitting network and the first branch network, and iterate until the fitting network and the first branch network converge.

[0066] The aforementioned local face dataset includes local face image samples and corresponding local face annotation data.

[0067] The following example uses the human eye dataset as a local face dataset. For training the fitting network and the first branch network on other types of local face datasets, the human eye dataset can be used as a reference.

[0068] One possible approach is to include human eye datasets comprising: human eye image samples, human eye bounding box annotations, and human eye keypoint annotations. The aforementioned human eye image samples can be face images including human eyes. The human eye image samples are input into a fitting network and a human eye detection network (i.e., the first branch network mentioned above) for convolutional computation, outputting human eye bounding box data and human eye keypoint data. A human eye bounding box loss function is calculated between the human eye bounding box data and the human eye bounding box annotations, and a human eye keypoint loss function is calculated between the human eye keypoint data and the human eye keypoint annotations. The total loss of the human eye bounding box loss function and the human eye keypoint loss function is used as the first loss function. This first loss function is backpropagated, and the parameters in the fitting network and the human eye detection network are adjusted using gradient descent. This process is iterated until the first loss function is minimized, converges in the human eye test dataset, or reaches a preset number of iterations. At this point, the human eye bounding box data and human eye keypoint data output by the fitting network and the human eye detection network are very close to the human eye bounding box data and human eye keypoint annotation data.

[0069] Another possible approach is to include human eye datasets comprising human eye image samples and human eye bounding box (HOB) annotations. The human eye image samples can be face images including human eyes. The human eye image samples are input into a fitting network and a human eye detection network (i.e., the first branch network mentioned above) for convolutional computation, outputting human eye bounding box data. The human eye bounding box loss function between the human eye bounding box data and the HOB annotations is calculated as the first loss function. This first loss function is backpropagated, and gradient descent is used to adjust the parameters in the fitting network and the human eye detection network. This process is iterated until the first loss function is minimized, converges in the human eye test dataset, or reaches a preset number of iterations. At this point, the human eye bounding box data output by the fitting network and the human eye detection network is very close to the human eye bounding box annotations.

[0070] Another possible approach is to include human eye datasets comprising human eye image samples and human eye keypoint annotation data. The aforementioned human eye image samples can be face images including human eyes. The human eye image samples are input into a fitting network and a human eye detection network (i.e., the first branch network mentioned above) for convolutional computation, outputting human eye keypoint data. The human eye keypoint loss function between the human eye keypoint data and the human eye keypoint annotation data is calculated as the first loss function. This first loss function is backpropagated, and gradient descent is used to adjust the parameters in the fitting network and the human eye detection network. This process is iterated until the first loss function is minimized, converges in the human eye test dataset, or reaches a preset number of iterations. At this point, the human eye keypoint data output by the fitting network and the human eye detection network is very close to the human eye keypoint annotation data.

[0071] In this embodiment of the invention, the fitting network and the first branch network are trained using a local face dataset, enabling the fitting network to extract local face features. This allows the local face features to be used as an auxiliary detection method for face detection, thereby improving the detection accuracy of occluded faces.

[0072] 302. Fix the parameters of the convergent fitting network, and train the convergent fitting network and the second branch network using a face dataset to obtain the convergent second branch network.

[0073] The face sample images in the aforementioned face dataset include local face images. Therefore, when training the convergent fitting network and the second branch network, the convergent fitting network uses extracted local face features from the face samples as prior features to assist in detection.

[0074] Optionally, the aforementioned face dataset includes face annotation data, which includes face bounding box annotation data and / or face landmark annotation data. The face bounding box annotation data corresponds to face detection, and the face landmark annotation data corresponds to face landmark detection.

[0075] The type of face annotation data in the above face dataset can be selected according to the specific application scenario of the face detection model. If only the faces in the face bounding boxes are regressed, the face annotation data can be face bounding box annotation data; if only facial landmark information is regressed, the face annotation data can be facial landmark annotation data; if both the faces in the face bounding boxes and facial landmark annotation data are regressed, the face annotation data can be both face bounding box annotation data and facial landmark annotation data.

[0076] Optionally, if the converged fitting network and the second branch network are trained using face bounding box annotation data, the calculated result is face bounding box information, such as eye boxes, ear boxes, eyebrow boxes, and mouth boxes. Of course, face bounding box information can also be understood as face position. The face bounding box information can be expressed as (x, y, h, w), where x and y are the coordinates of the center point of the face bounding box, h is the height of the face bounding box, and w is the width of the local face bounding box. If the converged fitting network and the second branch network are trained using facial landmark annotation data, the resulting calculations will contain facial landmark information. This information may include one or more of the following: eye landmarks, ear landmarks, eyebrow landmarks, mouth landmarks, and facial contour landmarks. Alternatively, facial landmark information can be understood as the location of key points on the face; for example, eye landmarks can be points on the eye contour, ear landmarks can be points on the ear contour, eyebrow landmarks can be points on the eyebrow contour, mouth landmarks can be points on the mouth contour, and facial contour landmarks can be points on the facial contour. If the converged fitting network and the second branch network are trained using both facial bounding box annotation data and facial landmark annotation data, the resulting calculations will include both facial bounding box information and facial landmark information.

[0077] The parameters of the aforementioned fixed convergent fitting network refer to the fact that the convergent fitting network will not adjust its parameters according to the backpropagation of the loss function during the training of the second branch network, thus maintaining the fitting network's ability to extract local facial features.

[0078] Specifically, for training the convergent fitting network and the second branch network, please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a flowchart of a convergent fitting network and a second branch network training method provided in an embodiment of the present invention, as shown below. Figure 5 As shown, it includes the following steps:

[0079] 501. Obtain the face dataset.

[0080] 502. Fix the parameters of the convergent fitting network.

[0081] 503. Input the face image sample into the converged fitting network and the second branch network for calculation to obtain the second calculation result, and calculate the second loss function between the second calculation result and the face annotation data.

[0082] 504. Perform backpropagation based on the second loss function to adjust the parameters of the second branch network, and iterate until the second branch network converges.

[0083] In this embodiment of the invention, the aforementioned face dataset includes face image samples and corresponding face annotation data. The face annotation data may include face bounding box annotation data and / or face local key point annotation data.

[0084] One possible approach is that the face dataset includes: face image samples, face bounding box annotation data, and face key point annotation data. The face image samples mentioned above are face images that include parts of the face, such as at least one local facial region including the eyes, ears, eyebrows, and mouth. Face image samples are input into a converged fitting network and a face detection network (i.e., the second branch network mentioned above) for convolution calculation, outputting face bounding box data and facial landmark data. The face bounding box loss function and the facial landmark loss function are calculated between the face bounding box data and the face bounding box annotation data. The total loss of the face bounding box loss function and the facial landmark loss function is used as the second loss function. The second loss function is backpropagated, and the parameters in the face detection network are adjusted using gradient descent. The above process is iterated until the second loss function is minimized, converges in the face test dataset, or reaches the preset number of iterations. At this point, the face bounding box data and facial landmark data output by the face detection network are very close to the face bounding box data and facial landmark annotation data.

[0085] Another possible approach is to include face image samples and face bounding box annotations in the face dataset. The face image samples can be face images including partial views of the face. The face image samples are input into a converged fitting network and a face detection network (i.e., the second branch network mentioned above) for convolutional computation, outputting face bounding box data. A second loss function is calculated between the face bounding box data and the face bounding box annotations. This second loss function is then backpropagated, and gradient descent is used to adjust the parameters in the face detection network. This process is iterated until the second loss function is minimized, converges on the face test dataset, or reaches a preset number of iterations. At this point, the face bounding box data output by the face detection network is very close to the face bounding box annotations.

[0086] Another possible approach is to include face image samples and facial landmark annotation data in the face dataset. The face image samples can be face images that include parts of the face. The face image samples are input into a converged fitting network and a face detection network (i.e., the second branch network mentioned above) for convolution calculation, outputting facial landmark data. The facial landmark loss function between the facial landmark data and the facial landmark annotation data is calculated as a second loss function. This second loss function is backpropagated, and the parameters in the face detection network are adjusted using gradient descent. This process is iterated until the second loss function is minimized, converges in the face test dataset, or reaches a preset number of iterations. At this point, the facial landmark data output by the face detection network is very close to the facial landmark annotation data.

[0087] In this embodiment of the invention, by fixing the parameters of the converged fitting network and training the second branch network with the assistance of the converged fitting network, the second branch network can perform face detection with the help of local facial features, thereby improving the detection accuracy of occluded faces.

[0088] 303. Based on the convergent fitting network and the convergent second branch network, a well-trained face detection model is obtained.

[0089] In this embodiment of the invention, the convergent fitting network and the convergent second branch network described above can directly constitute a face detection model.

[0090] In one possible embodiment, when detecting a face, only the face detection result can be output, instead of the detection result of a local face, thereby reducing the size of the face detection model and making it easier to deploy in embedded devices in a miniaturized manner.

[0091] After obtaining the converged fitting network and the converged first branch network, the converged first branch network can be pruned. Then, with the parameters of the converged fitting network fixed, the converged fitting network and the second branch network are trained using a face dataset to obtain the converged second branch network. Fine-tuning of the converged fitting network and the converged second branch network yields the trained face detection model. Alternatively, after obtaining the converged fitting network and the converged first branch network, the parameters of the converged fitting network are fixed, and the converged fitting network and the second branch network are trained using a face dataset. Fine-tuning of the converged fitting network, the converged first branch network, and the converged second branch network is then performed, and the converged first branch network is pruned to obtain the converged second branch network.

[0092] In this embodiment of the invention, an image to be detected is acquired, the image including an occluded face; the image is input into a pre-trained face detection model, and a face detection result is obtained by outputting the pre-trained face detection model; wherein, the pre-trained face detection model incorporates auxiliary detection based on local facial features, and the pre-trained face detection model is obtained through auxiliary training using a local facial dataset. By using a face detection model with local facial features as auxiliary detection, face detection is performed on the image to be detected, which is equivalent to performing prior detection on the local facial features before detecting the face, thus improving the detection accuracy of occluded faces.

[0093] It should be noted that the face detection method provided in this embodiment of the invention can be applied to devices such as mobile phones, monitors, computers, and servers that are capable of face detection.

[0094] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a face detection device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes:

[0095] Acquisition module 601 is used to acquire an image to be detected, the image to be detected including an occluded human face;

[0096] The detection module 602 is used to input the image to be detected into a pre-trained face detection model, and output the face detection result through the pre-trained face detection model;

[0097] The pre-trained face detection model incorporates auxiliary detection based on local facial features, and this pre-trained face detection model is obtained through auxiliary training using a local facial dataset.

[0098] Optionally, the untrained face detection model includes a fitting network, a first branch network, and a second branch network. The fitting network is connected to both the first and second branch networks. The first branch network is used to assist in the training of the fitting network so that the fitting network can fit the auxiliary detection of local facial features. The trained face detection model includes the fitting network and the second branch network.

[0099] Optional, such as Figure 7 As shown, the device further includes:

[0100] The first training module 603 is used to train the fitting network and the first branch network using a local face dataset until the fitting network and the first branch network converge, thus obtaining a converged fitting network and a converged first branch network.

[0101] The second training module 604 is used to fix the parameters of the converged fitting network, and to train the converged fitting network and the second branch network using a face dataset to obtain a converged second branch network. The face sample images in the face dataset include local face images.

[0102] The processing module 605 is used to obtain a trained face detection model based on the converged fitting network and the converged second branch network.

[0103] Optionally, the processing module 605 is further configured to prune the converged first branch network to obtain a trained face detection model, wherein the trained face detection model contains only the converged fitting network and the converged second branch network.

[0104] Optional, such as Figure 8 As shown, the first training module 603 includes:

[0105] The first acquisition unit 6031 is used to acquire a local face dataset, which includes local face image samples and corresponding local face annotation data.

[0106] The first calculation unit 6032 is used to input the local face image sample into the fitting network and the first branch network for calculation, obtain a first calculation result, and calculate a first loss function between the first calculation result and the local face annotation data;

[0107] The first adjustment unit 6033 is used to perform backpropagation according to the first loss function to adjust the parameters of the fitting network and the first branch network, and iterate until the fitting network and the first branch network converge.

[0108] Optionally, the face local annotation data includes face local bounding box annotation data and / or face local key point annotation data, the first calculation result includes face local bounding box data and / or face local key point data, the first loss function is the loss function between face local bounding box data and face local bounding box annotation data, or the first loss function is the loss function between face local key point data and face local key point annotation data, or the first loss function is the total loss function of the loss function between face local bounding box data and face local bounding box annotation data and the loss function between face local key point data and face local key point annotation data.

[0109] Optional, such as Figure 9 As shown, the second training module 604 includes:

[0110] The second acquisition unit 6041 is used to acquire a face dataset, which includes face image samples and corresponding face annotation data.

[0111] Fixed unit 6042 is used to fix the parameters of the convergent fitting network;

[0112] The second calculation unit 6043 is used to input the face image sample into the converged fitting network and the second branch network for calculation, obtain a second calculation result, and calculate a second loss function between the second calculation result and the face annotation data;

[0113] The second adjustment unit 6044 is used to perform backpropagation according to the second loss function to adjust the parameters of the second branch network and iterate until the second branch network converges.

[0114] Optionally, the facial local features are human eye features, and the pre-trained face detection model is obtained through auxiliary training using a human eye dataset.

[0115] It should be noted that the face detection device provided in this embodiment of the invention can be applied to devices such as mobile phones, monitors, computers, and servers that can perform face detection.

[0116] The face detection device provided in this embodiment of the invention can implement all the processes of the face detection method in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0117] See Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 10 As shown, it includes: a memory 1002, a processor 1001, and a computer program stored in the memory 1002 and executable on the processor 1001, wherein:

[0118] The processor 1001 is used to call the computer program stored in the memory 1002 and perform the following steps:

[0119] Acquire an image to be detected, the image to be detected including an occluded human face;

[0120] The image to be detected is input into a pre-trained face detection model, and the face detection result is obtained by outputting the pre-trained face detection model.

[0121] The pre-trained face detection model incorporates auxiliary detection based on local facial features, and this pre-trained face detection model is obtained through auxiliary training using a local facial dataset.

[0122] Optionally, the untrained face detection model includes a fitting network, a first branch network, and a second branch network. The fitting network is connected to both the first and second branch networks. The first branch network is used to assist in the training of the fitting network so that the fitting network can fit the auxiliary detection of local facial features. The trained face detection model includes the fitting network and the second branch network.

[0123] Optionally, the processor 1001 performs the following steps to train the untrained face detection model:

[0124] The fitting network and the first branch network are trained using a local face dataset until they converge, resulting in a converged fitting network and a converged first branch network.

[0125] With the parameters of the convergent fitting network fixed, the convergent fitting network and the second branch network are trained using a face dataset to obtain the convergent second branch network, wherein the face sample images in the face dataset include local face images.

[0126] Based on the converged fitting network and the converged second branch network, a well-trained face detection model is obtained.

[0127] Optionally, the processor 1001 executes the converged fitting network and the converged second branch network to obtain a trained face detection model, including:

[0128] The converged first branch network is pruned to obtain a trained face detection model, which contains only the converged fitting network and the converged second branch network.

[0129] Optionally, the process executed by processor 1001 to train the fitted network and the first branch network using a local face dataset includes:

[0130] Obtain a local face dataset, which includes local face image samples and corresponding local face annotation data;

[0131] The local face image sample is input into the fitting network and the first branch network for calculation to obtain a first calculation result, and a first loss function is calculated between the first calculation result and the local face annotation data.

[0132] Backpropagation is performed based on the first loss function to adjust the parameters of the fitting network and the first branch network, and iteration is performed until the fitting network and the first branch network converge.

[0133] Optionally, the face local annotation data includes face local bounding box annotation data and / or face local key point annotation data, the first calculation result includes face local bounding box data and / or face local key point data, the first loss function is the loss function between face local bounding box data and face local bounding box annotation data, or the first loss function is the loss function between face local key point data and face local key point annotation data, or the first loss function is the total loss function of the loss function between face local bounding box data and face local bounding box annotation data and the loss function between face local key point data and face local key point annotation data.

[0134] Optionally, the parameters of the fixed-convergence fitting network executed by the processor 1001 are used to train the converged fitting network and the second branch network using a face dataset, including:

[0135] Obtain a face dataset, which includes face image samples and corresponding face annotation data;

[0136] The parameters of the fitted network with fixed convergence are fixed;

[0137] The face image sample is input into the converged fitting network and the second branch network for calculation to obtain a second calculation result. The second loss function between the second calculation result and the face annotation data is calculated.

[0138] Backpropagation is performed based on the second loss function to adjust the parameters of the second branch network, and the process is iterated until the second branch network converges.

[0139] Optionally, the facial local features are human eye features, and the pre-trained face detection model is obtained through auxiliary training using a human eye dataset.

[0140] It should be noted that the aforementioned electronic devices can be mobile phones, monitors, computers, servers, and other devices that can perform face detection.

[0141] The electronic device provided in this embodiment of the invention can implement all the processes of the face detection method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0142] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the face detection method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0143] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0144] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A face detection method, characterized by, The method comprises the following steps: obtaining a to-be-detected image, wherein the to-be-detected image comprises an occluded face; inputting the to-be-detected image into a pre-trained face detection model, and outputting a face detection result through the pre-trained face detection model; wherein the pre-trained face detection model is fitted with auxiliary detection of a local feature of a face, and the pre-trained face detection model is obtained through auxiliary training of a local face data set; an untrained face detection model comprises a fitting network, a first branch network and a second branch network, wherein the fitting network is connected to the first branch network and the second branch network, the first branch network is used for assisting training of the fitting network, so that the fitting network fits auxiliary detection of a local feature of a face, and a trained face detection model comprises a fitting network and a second branch network; training the untrained face detection model to obtain a trained face detection model, comprising: training the fitting network and the first branch network through a local face data set until the fitting network and the first branch network converge, to obtain a converged fitting network and a converged first branch network; fixing parameters of the converged fitting network, training the converged fitting network and the second branch network through a face data set to obtain a converged second branch network, wherein the face data set comprises face sample images including local face images; obtaining a trained face detection model based on the converged fitting network and the converged second branch network.

2. The method of claim 1, wherein, obtaining a trained face detection model based on the converged fitting network and the converged second branch network, comprising: cutting the converged first branch network to obtain a trained face detection model, wherein the trained face detection model only comprises the converged fitting network and the converged second branch network.

3. The method of claim 1, wherein, training the fitting network and the first branch network through a local face data set, comprising: obtaining a local face data set, wherein the local face data set comprises local face image samples and corresponding local face annotation data; inputting the local face image samples into the fitting network and the first branch network for calculation to obtain a first calculation result, and calculating a first loss function between the first calculation result and the local face annotation data; performing back propagation according to the first loss function to adjust parameters of the fitting network and the first branch network, and iterating until the fitting network and the first branch network converge.

4. The method of claim 3, wherein, The face local annotation data includes face local frame annotation data and / or face local key point annotation data, the first calculation result includes face local frame data and / or face local key point data, the first loss function is a loss function between the face local frame data and the face local frame annotation data, or the first loss function is a loss function between the face local key point data and the face local key point annotation data, or the first loss function is a total loss function of a loss function between the face local frame data and the face local frame annotation data and a loss function between the face local key point data and the face local key point annotation data.

5. The method of claim 1, wherein, The parameters of the converged fitting network are fixed, and the converged fitting network and the second branch network are trained through a face data set, including: obtaining a face data set, the face data set including face image samples and corresponding face annotation data; fixing the parameters of the converged fitting network; inputting the face image samples into the converged fitting network and the second branch network for calculation to obtain a second calculation result, and calculating a second loss function between the second calculation result and the face annotation data; performing back propagation according to the second loss function to adjust the parameters of the second branch network, and iterating until the second branch network converges.

6. The method of any one of claims 1 to 5, wherein, The face local feature is an eye feature, and the pre-trained face detection model is obtained by auxiliary training through an eye data set.

7. A face detection apparatus characterized by comprising: The device includes: an acquisition module configured to acquire a to-be-detected image, the to-be-detected image including an occluded face; a detection module configured to input the to-be-detected image into a pre-trained face detection model, and output a face detection result through the pre-trained face detection model; wherein the pre-trained face detection model is fitted with auxiliary detection of a face local feature, and the pre-trained face detection model is obtained by auxiliary training through a face local data set; The untrained face detection model includes a fitting network, a first branch network and a second branch network, and the fitting network is connected with the first branch network and the second branch network, wherein the first branch network is used for assisting the training of the fitting network to make the fitting network fit the auxiliary detection of the face local feature, and the trained face detection model includes the fitting network and the second branch network. The detection module is configured to train the untrained face detection model to obtain the trained face detection model, and is specifically configured to: train the fitting network and the first branch network through a face local data set until the fitting network and the first branch network converge to obtain a converged fitting network and a converged first branch network; fix the parameters of the converged fitting network, train the converged fitting network and the second branch network through a face data set to obtain a converged second branch network, wherein the face sample images in the face data set include face local images; obtain the trained face detection model based on the converged fitting network and the converged second branch network.

8. An electronic device, comprising: including: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the face detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the face detection method according to any one of claims 1 to 6.

Images