Face detection deep learning network training method and device, equipment and medium

Abstract

The present application provides a kind of face detection deep learning network training method, device, equipment and medium, comprising: original face image is carried out image size transformation and obtains small size face image, small size face image is input into first deep learning network model, obtains N first prediction face frame information, N is positive integer;According to N first prediction face frame information, the size information of original face image and the size information of small size face image, calculate cutting area, obtain N cutting image;N cutting image is respectively input into second deep learning network model, obtains N second prediction face frame information;According to the size information of N cutting image and N second prediction face frame information, obtain the prediction face frame information of N original face image.The present application can effectively avoid the problem of inaccurate positioning and precision loss caused by size transformation, thereby improving the quality of face frame detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face detection, and in particular to a face detection deep learning network training method and device, equipment and medium. BACKGROUND

[0002] Current deep learning face recognition often transforms the size of the input image to a small size, and solves on the small size, and then converts to a large image through size transformation, which inevitably introduces conversion errors, causing inaccurate positioning, precision loss and other problems, and further causing low face frame detection quality. For example, if the original size of the input image is (3000, 4000) and the small image size is (256, 256), the theoretical error value is (±12, ±14), which will cause a serious reduction in detection quality.

[0003] Therefore, it is necessary to provide a face detection deep learning network training method, device, equipment and medium to solve the above problems. SUMMARY

[0004] The purpose of the present application is to provide a face detection deep learning network training method, device, equipment and medium to improve the problem of inaccurate positioning, precision loss and low detection quality caused by current deep learning face detection.

[0005] In a first aspect, the present application provides a face detection deep learning network training method, which comprises: performing image size transformation on an original face image to obtain a small size face image, inputting the small size face image into a first deep learning network model to obtain N first predicted face frame information, N being a positive integer; calculating a clipping region according to the N first predicted face frame information, size information of the original face image and size information of the small size face image to obtain N clipping images; inputting the N clipping images into a second deep learning network model respectively to obtain N second predicted face frame information; and obtaining predicted face frame information of the N original face images according to size information of the N clipping images and the N second predicted face frame information.

[0006] The method provided by the application has the beneficial effect that a cropped image prediction network is added, a cropped region is calculated according to first predicted face frame information of a small-size face image, size information of the original face image, and size information of the small-size face image, the original face image is cropped to obtain a cropped image, second predicted face frame information of the cropped image is obtained, and then the predicted face frame information of the original face image can be calculated. Since the predicted face frame information of the original face image in the application is not obtained through size transformation, but is detected based on the cropped image of the original face image, the problem of inaccurate positioning and precision loss caused by size transformation can be effectively avoided, thereby improving the quality of face frame detection.

[0007] In a possible embodiment, the small-size face image is input into the first deep learning network model to obtain N first predicted face frame information, including:

[0008] The small-size face image is input into the first deep learning network model to obtain M first predicted face frame information, M being a positive integer greater than N;

[0009] N first predicted face frame information with a size greater than a set threshold is selected from the M first predicted face frame information.

[0010] In a possible embodiment, after obtaining the N first predicted face frame information, the method further includes:

[0011] A first loss function is calculated through the small-size face image and the N first predicted face frame information to supervise training of the first deep learning network model.

[0012] In a possible embodiment, after obtaining the N second predicted face frame information, the method further includes:

[0013] A second loss function is calculated through the original face image and the N predicted face frame information of the original face image;

[0014] A third loss function is calculated through the N cropped images and the N second predicted face frame information;

[0015] Training of the second deep learning network model is supervised through the second loss function and / or the third loss function.

[0016] In a possible embodiment, the original face image is a single-face image or a multi-face image.

[0017] In a second aspect, the present application further provides a device for training a deep learning network for face detection. The device comprises modules / units for implementing the method of any of the possible designs of the first aspect. These modules / units can be implemented by hardware, or by hardware executing corresponding software.

[0018] In a third aspect, the present application provides an electronic device comprising a processor and a memory. The memory is configured to store one or more computer programs. When the one or more computer programs stored in the memory are executed by the processor, the electronic device is enabled to implement the method of any of the possible designs of the first aspect.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium comprising a computer program. When the computer program is run on an electronic device, the electronic device is enabled to implement the method of any of the possible designs of any of the above aspects.

[0020] In a fifth aspect, the present application provides a computer program product comprising a computer program. When the computer program product is run on an electronic device, the electronic device is enabled to implement the method of any of the possible designs of any of the above aspects.

[0021] The advantages of the above second to fifth aspects can be seen from the description of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 FIG. 1 is a flowchart of a face detection method in the prior art.

[0023] Figure 2 FIG. 2 is a flowchart of a method for training a deep learning network for face detection.

[0024] Figure 3 FIG. 3 is a schematic diagram of a device for training a deep learning network for face detection.

[0025] Figure 4 FIG. 4 is a schematic diagram of an electronic device. DETAILED DESCRIPTION

[0026] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are some of the embodiments of the present application but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the common meanings thereof by those skilled in the art. The similar words such as "comprise" used herein mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, and do not exclude other elements or objects.

[0027] Face detection in the prior art refers to finding the face position in an image given, usually using a rectangular frame, that is, the input is an image img, and the output is the rectangular frame information (x, y, w, h) of the face, including the starting x coordinate, the starting y coordinate, the width of the rectangular frame and the height of the rectangular frame.

[0028] The existing face frame detection implementation scheme is generally as shown in Figure 1 First, the large image img1 (with a size of W*H) is converted into a small image img2 (with a size of w*h) by size transformation, and the small image img2 and other required auxiliary information are sent into a deep learning network net1. The small image face frame information img2_pre is predicted on the small image img2 by the deep learning network net1. The small image face frame information is converted into large image face frame information img1_pre_rs by size transformation. Since this scheme predicts on the small image and then converts to the large image by size transformation, conversion errors are inevitably introduced, resulting in a decrease in the quality of face frame detection.

[0029] In view of the problems in the prior art, the embodiments of the present application provide a deep learning network training method for face detection, which can be hardened in a chip or a chip module, or used in a software algorithm in a mobile terminal, a computer terminal and other devices requiring camera imaging. Figure 2 The flowchart of the deep learning network training method for face detection of the present application is shown in Figure 2 The method comprises the following steps:

[0030] S201: performing image size transformation on an original face image to obtain a small-size face image, inputting the small-size face image into a first deep learning network model to obtain N first predicted face frame information, N being a positive integer.

[0031] In a preferred embodiment, the image size transformation can be a nearest neighbor interpolation, a bilinear interpolation, a cubic interpolation, or other common image size conversion method.

[0032] In a specific embodiment, the first deep learning network model can be a common CNN (Convolutional Neural Network) structure, such as unet, cascade structure, resnet (Residual Networks), or other model structures that can realize face detection.

[0033] S202: Calculate the cropping region according to the N first predicted face frame information, the size information of the original face image, and the size information of the small-size face image, and obtain N cropped images.

[0034] S203: Input the N cropped images into the second deep learning network model respectively, and obtain N second predicted face frame information.

[0035] In a specific embodiment, the second deep learning network model can have the same structure as the first deep learning network model, or can have a different structure. When the structures are different, the second deep learning network model can be a common CNN structure, such as unet, cascade structure, resnet, or other model structures that can realize face detection.

[0036] S204: Obtain the predicted face frame information of the N original face images according to the size information of the N cropped images and the N second predicted face frame information.

[0037] In this embodiment, the original face image is a single face image or a multi-face image. In a preferred embodiment, when the original face image is a multi-face image, the calculation is performed for each first predicted face frame information.

[0038] In another preferred embodiment, when the original face image is a multi-face image, the small-size face image is input into the first deep learning network model to obtain N first predicted face frame information, including: inputting the small-size face image into the first deep learning network model to obtain M first predicted face frame information, M being a positive integer greater than N, and selecting N first predicted face frame information with a size greater than a set threshold from the M first predicted face frame information. The key N first predicted face frame information is selected from the M first predicted face frame information for calculation, and the face frame information with a smaller size is not processed, which can further reduce the calculation time.

[0039] In addition, when the application is applied to a multi-face scene, the multi-face detection map is input into the second deep learning network model multiple times. Since the original size of the image is fixed, compared with the single-face scene, although the number of images input into the second deep learning network model is increased in the multi-face scene, the size of the input image is correspondingly reduced and a screening strategy can be added to select a key face frame, so that the time consumption will not increase too much, and the time consumption and the calculation cost will be obviously better than those of the existing face detection scheme.

[0040] In a preferred embodiment, after obtaining the N first predicted face frame information, the method further comprises: calculating a first loss function loss1 through the small-size face image and the N first predicted face frame information, and supervising the training of the first deep learning network model according to the first loss function loss1.

[0041] In a preferred embodiment, after obtaining the N second predicted face frame information, the method further comprises: calculating a second loss function loss2 through the original face image and the predicted face frame information of the N original face images, calculating a third loss function loss3 through the N cropped images and the N second predicted face frame information, and supervising the training of the second deep learning network model according to the second loss function loss2 and / or the third loss function loss3.

[0042] In a specific embodiment, in the process of training, the first loss function loss1, the second loss function loss2 and the third loss function loss3 can exist simultaneously, or only one or two of them can exist. The loss can be a single loss such as l2Loss and l1Loss, or a combined loss. The l2Loss is a mean square error (MSE), which is the most commonly used regression loss function. It is the square sum of the difference between the target variable and the predicted value. The l1Loss is a mean absolute error (MAE), which is another loss function for a regression model. The MAE is the sum of the absolute difference between the target variable and the predicted variable.

[0043] In a preferred embodiment, the calculation method of the cropped region can use the following formula:

[0044] x3=w1 / w2*(x2-a),x2-a≥0;y3=h1 / h2*(y2-b),y2-b≥0;

[0045] w3 = w1 / w2*(w2+a+c), w2+a+c≤w1; h3 = w1 / w2*(h2+b+d), h2+b+d≤h2; wherein, (x2, y2, w2, h2) is the first predicted face frame information; (w1, h1) is the size information of the original face image; (w2, h2) is the size information of the small size face image; (w3, h3) is the size information of the cropped image; (x3, y3) is the cropping starting coordinate information; a, b, c, d are parameters, usually taking 1. In addition, the information of the cropping region can also be obtained by other calculation methods.

[0046] In a preferred embodiment, the predicted face frame information of the original face image satisfies the following formula:

[0047] x1_p = x3_p+x3, x3_p+x3≤w1; y1_p = y3_p+y3, y3_p+y3≤h1;

[0048] w1_p = w3_p; h1_p = h3_p; wherein, (x3, y3, w3, h3) is the information of the cropping region; (x3_p, y3_p, w3_p, h3_p) is the second predicted face frame information; (x1_p, y1_p, w1_p, h1_p) is the predicted face frame information of the original face image. In addition, the predicted face frame information of the original face image can also be obtained by other calculation methods.

[0049] In a specific embodiment, the small size face image is input into the first deep learning network model, and the preprocessing map, hotspot map, and skin color detection map of the small size face image are input into the first deep learning network model.

[0050] In a specific embodiment, the cropped image is input into the second deep learning network model, and the preprocessing map, hotspot map, and skin color detection map of the cropped image are input into the second deep learning network model.

[0051] In the deep learning network training method for face detection in the embodiment, the first predicted face frame information of the small-size face image, the size information of the original face image, and the size information of the small-size face image are used to calculate a clipping region, a clipping image is obtained by clipping the original face image according to the clipping region, the second predicted face frame information of the clipping image is obtained by face detection, and the predicted face frame information of the original face image is obtained by calculation. Since the second predicted face frame information is obtained by recognizing and detecting the clipping image clipped from the original face image, the problem of inaccurate positioning and precision loss caused by size transformation can be avoided, and therefore the quality of face detection can be effectively improved. The deep learning network for face detection in the embodiment has an additional prediction network for the clipping image, and the neural network for predicting small image information can be designed to be more compact than the existing face detection network, and the problem of time consumption increase can be effectively avoided.

[0052] In addition, the deep learning network training device for face detection is also disclosed in the embodiment, Figure 3 FIG. 1 is a schematic diagram of the deep learning network training device for face detection in the embodiment, and FIG. 2 is a schematic diagram of the deep learning network training device for face detection in the embodiment. Figure 3 The device comprises a first face frame prediction unit 301 configured to perform image size transformation on an original face image to obtain a small-size face image, input the small-size face image into a first deep learning network model, and obtain N first predicted face frame information, N being a positive integer; a clipping unit 302 configured to calculate a clipping region according to the N first predicted face frame information, size information of the original face image, and size information of the small-size face image, and obtain N clipping images; a second face frame prediction unit 303 configured to input the N clipping images into a second deep learning network model respectively, and obtain N second predicted face frame information; and a third face frame prediction unit 304 configured to obtain N predicted face frame information of the original face image according to size information of the N clipping images and the N second predicted face frame information. All related contents of the steps involved in the method embodiment can be cited from the function description of the corresponding function module, and will not be described here.

[0053] In some other embodiments of the present application, the electronic device is disclosed in the embodiment, Figure 4 FIG. 3 is a structural schematic diagram of the electronic device in the embodiment, and FIG. 4 is a structural schematic diagram of the electronic device in the embodiment. Figure 4 The electronic device can comprise one or more processors 401, a memory 402, a display 403, one or more application programs (not shown), and one or more computer programs 404. The devices can be connected through one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 comprise instructions that can be used to perform the steps of the method embodiments, such asFigure 2 and Figure 3 and respective embodiments.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0055] The functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0056] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or said part that makes contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0057] Although the embodiments of the present application are described in detail above, it is obvious for those skilled in the art that various modifications and changes can be made to the embodiments. However, it should be understood that such modifications and changes all belong to the scope and spirit of the present application described in the claims. Moreover, the present application described herein can have other embodiments, and can be implemented or realized in various ways.

Claims

1. A method for training a deep learning network for face detection, characterized in that, The method includes: The original face image is resized to obtain a smaller face image. The smaller face image is then input into the first deep learning network model to obtain N first predicted face bounding boxes, where N is a positive integer. The cropping region is calculated based on N first predicted face bounding box information, the size information of the original face image, and the size information of the small face image, resulting in N cropped images; The N cropped images are input into the second deep learning network model to obtain N second predicted face bounding boxes. Based on the size information of the N cropped images and the N second predicted face bounding box information, the predicted face bounding box information of the N original face images is obtained.

2. The method according to claim 1, characterized in that, A small-sized face image is input into the first deep learning network model to obtain N first predicted face bounding boxes, including: A small face image is input into the first deep learning network model to obtain M first predicted face bounding boxes, where M is a positive integer greater than N; Select N first predicted face bounding boxes whose size is greater than a set threshold from the M first predicted face bounding boxes.

3. The method according to claim 2, characterized in that, After obtaining N first predicted face bounding boxes, the following is also included: The training of the first deep learning network model is supervised by calculating a first loss function using a small-sized face image and N first predicted face bounding boxes.

4. The method according to any one of claims 1 to 3, characterized in that, After obtaining N second predicted face bounding boxes, the following is also included: The second loss function is calculated using the original face image and the predicted face bounding box information of N original face images; The third loss function is calculated using N cropped images and N second predicted face bounding box information; The training of the second deep learning network model is supervised according to the second loss function and / or the third loss function.

5. The method according to any one of claims 1 to 3, characterized in that, The original face image can be a single face image or a multi-face image.

6. A deep learning network training device for face detection, characterized in that, The device includes: The first face bounding box prediction unit is used to transform the image size of the original face image to obtain a smaller face image. The smaller face image is then input into the first deep learning network model to obtain N first predicted face bounding box information, where N is a positive integer. The cropping unit is used to calculate the cropping region based on N first predicted face bounding box information, the size information of the original face image, and the size information of the small face image, to obtain N cropped images; The second face bounding box prediction unit is used to input the N cropped images into the second deep learning network model to obtain N second predicted face bounding box information. The third face bounding box prediction unit is used to obtain the predicted face bounding box information of the N original face images based on the size information of the N cropped images and the N second predicted face bounding box information.

7. The apparatus according to claim 6, characterized in that, Also includes: The filtering unit is used to input a small-sized face image into a first deep learning network model to obtain M first predicted face bounding boxes, where M is a positive integer greater than N, and to filter N first predicted face bounding boxes whose size is greater than a set threshold from the M first predicted face bounding boxes.

8. The apparatus according to claim 7, characterized in that, Also includes: The first training supervision unit is used to calculate the first loss function using a small-sized face image and N first predicted face bounding boxes, thereby supervising the training of the first deep learning network model.

9. The apparatus according to any one of claims 6 to 8, characterized in that, Also includes: The second training supervision unit is used to calculate a second loss function using the original face image and the predicted face bounding box information of N original face images, calculate a third loss function using the N cropped images and the N predicted face bounding box information, and supervise the training of the second deep learning network model based on the second loss function and / or the third loss function.

10. The apparatus according to any one of claims 6 to 8, characterized in that, The original face image can be a single face image or a multi-face image.

11. An electronic device, characterized in that, include: Processor and memory, wherein the memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory to cause the electronic device to perform the method according to any one of claims 1 to 5.

12. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

Images