Face detection method and device, computer readable storage medium and electronic equipment

Abstract

The invention belongs to the technical field of face detection, and particularly relates to a face detection method and device, a computer readable storage medium and electronic equipment. The method comprises the following steps: performing face detection in a target image by using a preset face detection model to obtain a face detection result of the target image; wherein a re-parameterization module of the face detection model in the training process comprises N convolution kernels corresponding to faces of different scales, and N is an integer greater than or equal to 2. According to the face detection method and device, in the training process of the face detection model, different convolution kernels in the re-parameterization module can correspond to faces of different scales respectively, so that richer and more diversified face semantic features can be extracted by different convolution kernels, the performance of the face detection model is effectively improved, and a more accurate face detection result is obtained.

Description

Technical Field

[0001] The present application belongs to the field of face detection technology, and in particular, relates to a face detection method, device, computer-readable storage medium and electronic device. Background Art

[0002] Face detection is a computer technology that finds the position and size of a face in any digital image, as well as the precise location of subtle facial features such as eyes, nose, mouth, etc. It is the basis for subsequent face recognition and analysis.

[0003] In the prior art, the convolution re-parameterization algorithm can be applied to the face detection model. The main idea is to increase the number of convolution kernels during training and fuse the parameters of the corresponding convolution kernels during inference, thereby improving the performance of the face detection model without increasing the inference consumption. However, the existing convolution re-parameterization algorithm extracts relatively limited facial semantic features during the training process, and has a poor effect on improving the performance of the face detection model. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a face detection method, device, computer-readable storage medium and electronic device to solve the problem that the existing convolution re-parameterization algorithm extracts relatively limited facial semantic features during the training process, and has a poor performance improvement effect on the face detection model.

[0005] A first aspect of an embodiment of the present application provides a face detection method, which may include:

[0006] Acquire the target image to be detected;

[0007] Performing face detection in the target image using a preset face detection model to obtain a face detection result of the target image;

[0008] Among them, the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2.

[0009] In a specific implementation of the first aspect, the training process of the face detection model may include:

[0010] Get the preset face detection training sample set;

[0011] Dividing the face detection frames in the face detection training sample set into N detection frame groups corresponding to faces of different scales; wherein the nth detection frame group corresponds to the nth convolution kernel in the re-parameterization module, 1≤n≤N;

[0012] In each training batch, determine the training loss for each of the N detection box groups.

[0013] The face detection model is trained according to the training losses of the N detection frame groups to obtain a trained face detection model.

[0014] In a specific implementation of the first aspect, the training of the face detection model according to the training loss of the N detection frame groups may include:

[0015] For the target convolution kernel, respectively determine the gradient of the training loss of the N detection frame groups; wherein the target convolution kernel is any convolution kernel in the reparameterization module;

[0016] Determine a total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups; wherein the gradient weight of the first detection frame group is greater than the gradient weight of the second detection frame group, the first detection frame group is the detection frame group corresponding to the target convolution kernel, and the second detection frame group is the other detection frame group except the first detection frame group;

[0017] According to the total gradient corresponding to the target convolution kernel, the model parameters of the target convolution kernel are adjusted.

[0018] In a specific implementation manner of the first aspect, the re-parameterization module may include a first convolution kernel, a second convolution kernel, and a third convolution kernel, the face scale corresponding to the first convolution kernel is larger than the face scale corresponding to the second convolution kernel, and the face scale corresponding to the second convolution kernel is larger than the face scale corresponding to the third convolution kernel;

[0019] Determining the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups may include:

[0020] When the target convolution kernel is the first convolution kernel, determining a first weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a first gradient weight of the first detection frame group; wherein the first detection frame group is a detection frame group corresponding to the first convolution kernel;

[0021] Determine a first weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the first gradient weight of the second detection frame group; wherein the second detection frame group is a detection frame group corresponding to the second convolution kernel;

[0022] Determining a first weighted gradient of the third detection frame grouping according to a gradient of a training loss of the third detection frame grouping and a first gradient weight of the third detection frame grouping; wherein the third detection frame grouping is a detection frame grouping corresponding to the third convolution kernel;

[0023] Determine a total gradient corresponding to the first convolution kernel according to the first weighted gradient of the first detection box group, the first weighted gradient of the second detection box group, and the first weighted gradient of the third detection box group.

[0024] In a specific implementation of the first aspect, determining the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups may also include:

[0025] When the target convolution kernel is the second convolution kernel, determining a second weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a second gradient weight of the first detection frame group;

[0026] Determine a second weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the second gradient weight of the second detection frame group;

[0027] Determine a second weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the second gradient weight of the third detection frame group;

[0028] Determine a total gradient corresponding to the second convolution kernel according to the second weighted gradient of the first detection box group, the second weighted gradient of the second detection box group, and the second weighted gradient of the third detection box group.

[0029] In a specific implementation of the first aspect, determining the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups may also include:

[0030] When the target convolution kernel is the third convolution kernel, determining a third weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a third gradient weight of the first detection frame group;

[0031] Determine a third weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the third gradient weight of the second detection frame group;

[0032] Determine a third weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the third gradient weight of the third detection frame group;

[0033] Determine a total gradient corresponding to the third convolution kernel according to the third weighted gradient of the first detection frame group, the third weighted gradient of the second detection frame group, and the third weighted gradient of the third detection frame group.

[0034] In a specific implementation manner of the first aspect, the convolution kernel corresponding to the minimum-scale face in the re-parameterization module may be a center difference convolution kernel.

[0035] A second aspect of an embodiment of the present application provides a face detection device, which may include:

[0036] A target image acquisition module is used to acquire a target image to be detected;

[0037] A face detection module, used to perform face detection in the target image using a preset face detection model to obtain a face detection result of the target image;

[0038] Among them, the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2.

[0039] In a specific implementation of the second aspect, the face detection device may further include:

[0040] A training sample set acquisition module is used to acquire a preset face detection training sample set;

[0041] A detection frame grouping module, used to divide the face detection frames in the face detection training sample set into N detection frame groups corresponding to faces of different scales; wherein the nth detection frame group corresponds to the nth convolution kernel in the re-parameterization module, 1≤n≤N;

[0042] A training loss determination module is used to determine the training losses of N detection box groups in each training batch;

[0043] The model training module is used to train the face detection model according to the training losses of the N detection frame groups to obtain a trained face detection model.

[0044] In a specific implementation of the second aspect, the model training module may include:

[0045] A gradient determination unit, used to determine the gradient of the training loss of N detection frame groups for the target convolution kernel, respectively; wherein the target convolution kernel is any convolution kernel in the reparameterization module;

[0046] a total gradient determining unit, configured to determine a total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups; wherein the gradient weight of the first detection frame group is greater than the gradient weight of the second detection frame group, the first detection frame group is the detection frame group corresponding to the target convolution kernel, and the second detection frame group is the detection frame group other than the first detection frame group;

[0047] A parameter adjustment unit is used to adjust the model parameters of the target convolution kernel according to the total gradient corresponding to the target convolution kernel.

[0048] In a specific implementation manner of the second aspect, the re-parameterization module may include a first convolution kernel, a second convolution kernel, and a third convolution kernel, the face scale corresponding to the first convolution kernel is larger than the face scale corresponding to the second convolution kernel, and the face scale corresponding to the second convolution kernel is larger than the face scale corresponding to the third convolution kernel;

[0049] The total gradient determination unit may include:

[0050] a first total gradient determination subunit, configured to determine, when the target convolution kernel is the first convolution kernel, a first weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a first gradient weight of the first detection frame group; wherein the first detection frame group is a detection frame group corresponding to the first convolution kernel; determine a first weighted gradient of the second detection frame group according to a gradient of a training loss of the second detection frame group and a first gradient weight of the second detection frame group; wherein the second detection frame group is a detection frame group corresponding to the second convolution kernel; determine a first weighted gradient of the third detection frame group according to a gradient of a training loss of the third detection frame group and a first gradient weight of the third detection frame group; wherein the third detection frame group is a detection frame group corresponding to the third convolution kernel; determine a total gradient corresponding to the first convolution kernel according to the first weighted gradient of the first detection frame group, the first weighted gradient of the second detection frame group, and the first weighted gradient of the third detection frame group.

[0051] In a specific implementation of the second aspect, the total gradient determining unit may further include:

[0052] The second total gradient determination subunit is used to determine the second weighted gradient of the first detection frame group according to the gradient of the training loss of the first detection frame group and the second gradient weight of the first detection frame group when the target convolution kernel is the second convolution kernel; determine the second weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the second gradient weight of the second detection frame group; determine the second weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the second gradient weight of the third detection frame group; determine the total gradient corresponding to the second convolution kernel according to the second weighted gradient of the first detection frame group, the second weighted gradient of the second detection frame group, and the second weighted gradient of the third detection frame group.

[0053] In a specific implementation of the second aspect, the total gradient determining unit may further include:

[0054] The third total gradient determination subunit is used to determine, when the target convolution kernel is the third convolution kernel, a third weighted gradient of the first detection frame group according to the gradient of the training loss of the first detection frame group and the third gradient weight of the first detection frame group; determine the third weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the third gradient weight of the second detection frame group; determine the third weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the third gradient weight of the third detection frame group; determine the total gradient corresponding to the third convolution kernel according to the third weighted gradient of the first detection frame group, the third weighted gradient of the second detection frame group, and the third weighted gradient of the third detection frame group.

[0055] In a specific implementation manner of the second aspect, the convolution kernel corresponding to the minimum-scale face in the re-parameterization module may be a center difference convolution kernel.

[0056] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned face detection methods are implemented.

[0057] A fourth aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned face detection methods when executing the computer program.

[0058] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of any one of the above-mentioned face detection methods.

[0059] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain a target image to be detected; use a preset face detection model to perform face detection in the target image, and obtain a face detection result of the target image; wherein the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2. Through the embodiments of the present application, during the training process of the face detection model, different convolution kernels in the re-parameterization module can be respectively corresponded to faces of different scales, so that different convolution kernels can extract richer and more diverse facial semantic features, thereby effectively improving the performance of the face detection model and obtaining more accurate face detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 Schematic flow chart of the training process of the face detection model in the embodiment of the present application;

[0062] Figure 2 A schematic diagram of dividing a face detection frame into detection frame groups corresponding to faces of different scales;

[0063] Figure 3 Schematic diagram of extracting facial semantic features using conventional convolution kernels;

[0064] Figure 4 Schematic diagram of facial semantic feature extraction using multi-scale convolution kernels;

[0065] Figure 5 This is a schematic flow chart of an embodiment of a face detection method in the embodiments of the present application;

[0066] Figure 6 This is a structural diagram of an embodiment of a face detection device in an embodiment of the present application;

[0067] Figure 7 This is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0070] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0071] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0072] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0073] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0074] Face detection is a computer technology that is responsible for finding the position and size of a face in any digital image, as well as locating subtle facial features such as the precise position of the eyes, nose, mouth, etc. It is the basis for subsequent face recognition and analysis.

[0075] In the prior art, the convolution re-parameterization algorithm can be applied to the face detection model. The main idea is to increase the number of convolution kernels during training and fuse the parameters of the corresponding convolution kernels during inference, thereby improving the performance of the face detection model without increasing the inference consumption. However, the existing convolution re-parameterization algorithm extracts relatively limited facial semantic features during the training process, and has a poor effect on improving the performance of the face detection model.

[0076] In view of this, the embodiments of the present application provide a face detection method, device, computer-readable storage medium and electronic device to solve the problem that the existing convolution re-parameterization algorithm extracts relatively limited facial semantic features during the training process, and has a poor performance improvement effect on the face detection model.

[0077] The execution subject of the embodiment of the present application may be an electronic device, which may include but is not limited to computing devices such as mobile phones, tablet computers, desktop computers, notebooks, PDAs, robots, and servers. It should be noted that the electronic device for model training and the electronic device for face detection may be the same electronic device or different electronic devices. That is, the model training may be performed in one electronic device, and then the trained model may be deployed in another electronic device for face detection.

[0078] In the embodiment of the present application, the specific model structure adopted by the face detection model can be flexibly set according to actual conditions, and may include but is not limited to active shape models (ASM), active appearance models (AAM), cascaded pose regression (CPR), deep convolutional networks (DCNN), task-constrained deep convolutional networks (TCDCN), multi-task cascaded convolutional networks (MTCNN), tweaked convolutional neural networks (TCNN), DAN (Deep Alignment Networks) and other model structures. At least one of the above model structures is not specifically limited in the embodiment of the present application.

[0079] In the training process of the face detection model in the embodiment of the present application, different convolution kernels in the re-parameterization module can correspond to faces of different scales, so that different convolution kernels can extract richer and more diverse facial semantic features, thereby effectively improving the performance of the face detection model and obtaining more accurate face detection results. Here, the number of convolution kernels in the re-parameterization module is recorded as N, where N is an integer greater than or equal to 2, and its specific value can be flexibly set according to actual conditions, and the embodiment of the present application does not make specific limitations on this.

[0080] like Figure 1 As shown, in a specific implementation of the embodiment of the present application, the training process of the face detection model may include:

[0081] Step S101: Obtain a preset face detection training sample set.

[0082] In order to obtain a face detection model that can realize the face detection function, a preset face detection training sample set can be used to train the model, wherein the face detection training sample set can include sample images of faces of various scales, and each sample image is pre-labeled with a corresponding face detection frame.

[0083] Step S102: Divide the face detection frames in the face detection training sample set into N detection frame groups corresponding to faces of different scales.

[0084] Among them, the nth detection box group corresponds to the nth convolution kernel in the reparameterization module, 1≤n≤N.

[0085] In a specific implementation of the embodiment of the present application, N-1 area thresholds may be pre-set, which are denoted as AreaT, ... 1 AreaT 2 AreaT 3 、……、AreaT N-2 AreaT N-1 For any face detection frame, the area of ​​the face detection frame is recorded as Area. If Area>AreaT 1 , it can be divided into the first detection frame group, that is, the face detection frame group with the largest scale; if AreaT 2 <Area≤AreaT 1 , it can be divided into the second detection frame group, that is, the second largest scale face detection frame group; if AreaT 3 <Area≤AreaT 2 , it can be divided into the third detection frame group; ..., if AreaT N-1 <Area≤AreaT N-2, it can be divided into the N-1th detection frame group, that is, the next smallest scale face detection frame group; if Area≤AreaT N-1 , it can be divided into the Nth detection frame group, that is, the face detection frame group with the smallest scale. The specific value of each area threshold can be flexibly set according to the actual situation, and the embodiment of the present application does not make specific limitations on this.

[0086] Taking the case where N is 3 as an example, a first area threshold and a second area threshold can be preset, wherein the first area threshold is greater than the second area threshold. Figure 2 As shown, if the area of ​​the face detection frame is greater than the first area threshold, it can be divided into the first detection frame group, that is, the large-scale face detection frame group; if the area of ​​the face detection frame is less than or equal to the first area threshold and greater than the second area threshold, it can be divided into the second detection frame group, that is, the medium-scale face detection frame group; if the area of ​​the face detection frame is less than or equal to the second area threshold, it can be divided into the third detection frame group, that is, the small-scale face detection frame group.

[0087] Step S103: In each training batch, the training losses of the N detection box groups are determined respectively.

[0088] For any sample image, it can be input into the face detection model for processing, and the face detection result output by the face detection model can be obtained. According to the difference between the pre-labeled face detection frame and the actual face detection frame in the face detection result, the corresponding training loss can be determined.

[0089] In a specific implementation of the embodiment of the present application, the training losses of each face detection frame belonging to the same detection frame group may be summed to obtain the training loss of the detection frame group.

[0090] Step S104: train the face detection model according to the training losses of the N detection frame groups to obtain a trained face detection model.

[0091] The specific type of each convolution kernel in the re-parameterization module can be flexibly set according to the actual situation, and the embodiment of the present application does not specifically limit this. In a specific implementation of the embodiment of the present application, the convolution kernel corresponding to the smallest scale face can be a central difference convolution kernel (CDC), which can extract the difference between the current pixel value and the surrounding position, and can better extract edge information than ordinary convolution kernels, and is more suitable for extracting the features of small-scale faces. Other convolution kernels can be ordinary convolution kernels.

[0092] In the process of back propagation according to the training loss, taking any convolution kernel in the reparameterization module (denoted as the target convolution kernel) as an example, for the target convolution kernel, the gradients of the training losses of the N detection frame groups can be determined respectively, and the total gradient corresponding to the target convolution kernel can be determined according to the gradients of the training losses of the N detection frame groups and the preset gradient weights of the N detection frame groups.

[0093] Among them, the gradient weight of the first detection frame group is greater than the gradient weight of the second detection frame group, the first detection frame group is the detection frame group corresponding to the target convolution kernel, and the second detection frame group is the other detection frame group except the first detection frame group. The specific value of each gradient weight can be flexibly set according to the actual situation, and the embodiment of the present application does not make specific limitations on this.

[0094] After obtaining the total gradient corresponding to the target convolution kernel, the model parameters of the target convolution kernel can be adjusted according to the total gradient.

[0095] Taking the case where N is 3 as an example, the re-parameterization module may include a first convolution kernel, a second convolution kernel, and a third convolution kernel, wherein the face scale corresponding to the first convolution kernel is larger than the face scale corresponding to the second convolution kernel, and the face scale corresponding to the second convolution kernel is larger than the face scale corresponding to the third convolution kernel. Here, the detection frame group corresponding to the first convolution kernel can be recorded as the first detection frame group, the detection frame group corresponding to the second convolution kernel can be recorded as the second detection frame group, and the detection frame group corresponding to the third convolution kernel can be recorded as the third detection frame group.

[0096] When the target convolution kernel is the first convolution kernel, the first weighted gradient of the first detection frame group can be determined according to the gradient of the training loss of the first detection frame group and the first gradient weight of the first detection frame group, as shown in the following formula: w_grad 11 =grad 11 × 11 , where grad 11 The training loss of the first detection box group is the gradient of the first convolution kernel, w 11 The first gradient weight for the first detection box group. As an example, it can be set to 1, w_grad 11 The first weighted gradient for the first detection box group.

[0097] The first weighted gradient of the second detection frame group can be determined according to the gradient of the training loss of the second detection frame group and the first gradient weight of the second detection frame group, as shown in the following formula: 21 =grad 21 × 21 , where grad 21The training loss for the second detection box group is the gradient of the first convolution kernel, w 21 The first gradient weight for the second detection box group, as an example, it can be set to 0.5, w_grad 21 The first weighted gradient for the second detection box group.

[0098] The first weighted gradient of the third detection frame group may be determined according to the gradient of the training loss of the third detection frame group and the first gradient weight of the third detection frame group, as shown in the following formula: w_grad 31 =grad 31 × 31 , where grad 31 The training loss for the third detection box group is the gradient of the first convolution kernel, w 31 The first gradient weight for the third detection box group, as an example, can be set to 0.5, w_grad 31 The first weighted gradient for the third detection box group.

[0099] Finally, the total gradient corresponding to the first convolution kernel can be determined according to the first weighted gradient of the first detection frame group, the first weighted gradient of the second detection frame group, and the first weighted gradient of the third detection frame group, as shown in the following formula: w_grad 1 =w_grad 11 +w_grad 21 +w_grad 31 , where w_grad 1 is the total gradient corresponding to the first convolution kernel. After obtaining the total gradient corresponding to the first convolution kernel, the model parameters of the first convolution kernel can be adjusted according to the total gradient.

[0100] When the target convolution kernel is the second convolution kernel, the second weighted gradient of the first detection frame group can be determined according to the gradient of the training loss of the first detection frame group and the second gradient weight of the first detection frame group, as shown in the following formula: w_grad 12 =grad 12 × 12 , where grad 12 The training loss of the first detection box group is the gradient of the second convolution kernel, w 12 The second gradient weight for the first detection box group, as an example, can be set to 0.5, w_grad 12 A second weighted gradient for the first detection box group.

[0101] The second weighted gradient of the second detection frame group may be determined according to the gradient of the training loss of the second detection frame group and the second gradient weight of the second detection frame group, as shown in the following formula:22 =grad 22 × 22 , where grad 22 The training loss for the second detection box group is the gradient of the second convolution kernel, w 22 The second gradient weight for the second detection box group, as an example, it can be set to 1, w_grad 22 A second weighted gradient for a second detection box group.

[0102] The second weighted gradient of the third detection frame group may be determined according to the gradient of the training loss of the third detection frame group and the second gradient weight of the third detection frame group, as shown in the following formula: w_grad 32 =grad 32 × 32 , where grad 32 The training loss for the third detection box group is the gradient of the second convolution kernel, w 32 The second gradient weight for the third detection box group, as an example, can be set to 0.3, w_grad 32 A second weighted gradient for a third detection box group.

[0103] Finally, the total gradient corresponding to the second convolution kernel can be determined according to the second weighted gradient of the first detection frame group, the second weighted gradient of the second detection frame group, and the second weighted gradient of the third detection frame group, as shown in the following formula: w_grad 2 =w_grad 12 +w_grad 22 +w_grad 32 , where w_grad 2 is the total gradient corresponding to the second convolution kernel. After obtaining the total gradient corresponding to the second convolution kernel, the model parameters of the second convolution kernel can be adjusted according to the total gradient.

[0104] When the target convolution kernel is the third convolution kernel, the third weighted gradient of the first detection frame group can be determined according to the gradient of the training loss of the first detection frame group and the third gradient weight of the first detection frame group, as shown in the following formula: w_grad 13 =grad 13 × 13 , where grad 13 The training loss of the first detection box group is the gradient of the third convolution kernel, w 13 The third gradient weight for the first detection box group, as an example, can be set to 0.1, w_grad 13 The third weighted gradient for the first detection box group.

[0105] According to the gradient of the training loss of the second detection frame group and the third gradient weight of the second detection frame group, the third weighted gradient of the second detection frame group is determined, as shown in the following formula: w_grad 23 =grad 23 × 23 , where grad 23 The training loss for the second detection box group is the gradient of the third convolution kernel, w 23 The third gradient weight for the second detection box group, as an example, can be set to 0.1, w_grad 23 The third weighted gradient for the second detection box group.

[0106] According to the gradient of the training loss of the third detection frame group and the third gradient weight of the third detection frame group, the third weighted gradient of the third detection frame group is determined, as shown in the following formula: w_grad 33 =grad 33 × 33 , where grad 33 The training loss for the third detection box group is the gradient of the third convolution kernel, w 33 The third gradient weight for the third detection box group, as an example, it can be set to 1, w_grad 33 The third weighted gradient for the third detection box group.

[0107] Finally, the total gradient corresponding to the third convolution kernel can be determined according to the third weighted gradient of the first detection frame group, the third weighted gradient of the second detection frame group, and the third weighted gradient of the third detection frame group, as shown in the following formula: w_grad 3 =w_grad 13 +w_grad 23 +w_grad 33 , where w_grad 3 is the total gradient corresponding to the second convolution kernel. After obtaining the total gradient corresponding to the second convolution kernel, the model parameters of the second convolution kernel can be adjusted according to the total gradient.

[0108] Figure 3 The figure shows a schematic diagram of using a conventional convolution kernel to extract facial semantic features. As shown in the figure, the same processing is used for faces of different scales, and there is a lack of gradient differences between different convolution kernels, making it difficult to extract rich facial semantic features. Figure 4 The figure shows a schematic diagram of facial semantic feature extraction using multi-scale convolution kernels. As shown in the figure, by limiting the gradient flow of different convolution kernels in the reparameterization module for faces of different scales, different convolution kernels are allowed to focus on the features of faces of different scales, enabling them to extract richer facial semantic features, thereby better tapping the potential of reparameterized convolution.

[0109] It should be noted that the above content is to illustrate the adjustment process of model parameters by taking one training batch as an example. In the actual training process, multiple rounds of training can be performed, each round includes multiple training batches, and the above process is repeated until the preset training conditions are met, wherein the training condition can be that the training rounds reach a preset number threshold, and the number threshold can be set according to the actual situation, for example, it can be set to thousands, tens of thousands, hundreds of thousands or even larger values; the training condition can also be that the face detection model converges; since the training rounds may not reach the number threshold, but the face detection model has converged, unnecessary work may be repeated; or the face detection model may not converge, which may lead to an infinite loop and the training process cannot be ended. Based on the above two situations, the training condition can also be that the training rounds reach the number threshold or the face detection model converges. When the training conditions are met, a trained face detection model can be obtained.

[0110] Based on the trained face detection model, the structure is re-parameterized to obtain the face detection model for actual face detection, and the face detection model is used to perform actual face detection. Figure 5 As shown, an embodiment of a face detection method in an embodiment of the present application may include:

[0111] Step S501: Acquire a target image to be detected.

[0112] In a specific implementation of the embodiment of the present application, the target image can be directly acquired by the electronic device through its visual sensor. As an example, the electronic device can acquire a frame of image at a certain interval to form an image sequence or video stream. The type of image acquired can be set according to actual conditions, and can include but is not limited to RGB images, which is not specifically limited in the embodiment of the present application.

[0113] In another specific implementation of the embodiment of the present application, the target image can be acquired by other devices and sent to the electronic device through a preset data transmission link.

[0114] It should be noted that the "target image" mentioned in the present application refers to a frame of image currently being processed by the electronic device. It refers to a dynamic object rather than a specific frame of image. For example, the electronic device first processes the image acquired for the first time and records it as image 1, then the target image is image 1. After the electronic device processes image 1, it continues to process the image acquired for the second time and records it as image 2. Then the target image is image 2. After the electronic device processes image 2, it continues to process the image acquired for the third time and records it as image 3. Then the target image is image 3, and so on.

[0115] Step S502: Use the face detection model to perform face detection in the target image to obtain a face detection result of the target image.

[0116] In an embodiment of the present application, the target image can be input into a face detection model for processing, and the output of the face detection model can be obtained, and the output is the face detection result of the target image.

[0117] To sum up, through the embodiments of the present application, in the training process of the face detection model, different convolution kernels in the re-parameterization module can be respectively corresponded to faces of different scales, so that different convolution kernels can extract richer and more diverse facial semantic features, thereby effectively improving the performance of the face detection model and obtaining more accurate face detection results.

[0118] It should be noted that the information collection process (such as the face image collection process) / feature extraction process involved in this application is performed with the user's knowledge and permission, that is, the information collection process / feature extraction process complies with relevant requirements and does not constitute an act that harms the public interest.

[0119] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0120] Corresponding to the face detection method described in the above embodiment, Figure 6 A structural diagram of an embodiment of a face detection device provided in an embodiment of the present application is shown.

[0121] In this embodiment, a face detection device may include:

[0122] The target image acquisition module 601 is used to acquire the target image to be detected;

[0123] A face detection module 602, configured to perform face detection in the target image using a preset face detection model to obtain a face detection result of the target image;

[0124] Among them, the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2.

[0125] In a specific implementation of the embodiment of the present application, the face detection device may further include:

[0126] A training sample set acquisition module is used to acquire a preset face detection training sample set;

[0127] A detection frame grouping module, used to divide the face detection frames in the face detection training sample set into N detection frame groups corresponding to faces of different scales; wherein the nth detection frame group corresponds to the nth convolution kernel in the re-parameterization module, 1≤n≤N;

[0128] A training loss determination module is used to determine the training losses of N detection box groups in each training batch;

[0129] The model training module is used to train the face detection model according to the training losses of the N detection frame groups to obtain a trained face detection model.

[0130] In a specific implementation of the embodiment of the present application, the model training module may include:

[0131] A gradient determination unit, used to determine the gradient of the training loss of N detection frame groups for the target convolution kernel, respectively; wherein the target convolution kernel is any convolution kernel in the reparameterization module;

[0132] a total gradient determining unit, configured to determine a total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups; wherein the gradient weight of the first detection frame group is greater than the gradient weight of the second detection frame group, the first detection frame group is the detection frame group corresponding to the target convolution kernel, and the second detection frame group is the detection frame group other than the first detection frame group;

[0133] A parameter adjustment unit is used to adjust the model parameters of the target convolution kernel according to the total gradient corresponding to the target convolution kernel.

[0134] In a specific implementation of the embodiment of the present application, the re-parameterization module may include a first convolution kernel, a second convolution kernel, and a third convolution kernel, the face scale corresponding to the first convolution kernel is larger than the face scale corresponding to the second convolution kernel, and the face scale corresponding to the second convolution kernel is larger than the face scale corresponding to the third convolution kernel;

[0135] The total gradient determination unit may include:

[0136] a first total gradient determination subunit, configured to determine, when the target convolution kernel is the first convolution kernel, a first weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a first gradient weight of the first detection frame group; wherein the first detection frame group is a detection frame group corresponding to the first convolution kernel; determine a first weighted gradient of the second detection frame group according to a gradient of a training loss of the second detection frame group and a first gradient weight of the second detection frame group; wherein the second detection frame group is a detection frame group corresponding to the second convolution kernel; determine a first weighted gradient of the third detection frame group according to a gradient of a training loss of the third detection frame group and a first gradient weight of the third detection frame group; wherein the third detection frame group is a detection frame group corresponding to the third convolution kernel; determine a total gradient corresponding to the first convolution kernel according to the first weighted gradient of the first detection frame group, the first weighted gradient of the second detection frame group, and the first weighted gradient of the third detection frame group.

[0137] In a specific implementation of the embodiment of the present application, the total gradient determination unit may further include:

[0138] The second total gradient determination subunit is used to determine the second weighted gradient of the first detection frame group according to the gradient of the training loss of the first detection frame group and the second gradient weight of the first detection frame group when the target convolution kernel is the second convolution kernel; determine the second weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the second gradient weight of the second detection frame group; determine the second weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the second gradient weight of the third detection frame group; determine the total gradient corresponding to the second convolution kernel according to the second weighted gradient of the first detection frame group, the second weighted gradient of the second detection frame group, and the second weighted gradient of the third detection frame group.

[0139] In a specific implementation of the embodiment of the present application, the total gradient determination unit may further include:

[0140] The third total gradient determination subunit is used to determine, when the target convolution kernel is the third convolution kernel, a third weighted gradient of the first detection frame group according to the gradient of the training loss of the first detection frame group and the third gradient weight of the first detection frame group; determine the third weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the third gradient weight of the second detection frame group; determine the third weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the third gradient weight of the third detection frame group; determine the total gradient corresponding to the third convolution kernel according to the third weighted gradient of the first detection frame group, the third weighted gradient of the second detection frame group, and the third weighted gradient of the third detection frame group.

[0141] In a specific implementation of the embodiment of the present application, the convolution kernel corresponding to the minimum-scale face in the re-parameterization module may be a center difference convolution kernel.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0145] like Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned face detection method embodiments are implemented, for example Figure 5 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of modules 601 to 602 are shown.

[0146] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the electronic device 7.

[0147] The electronic device 7 may include but is not limited to mobile phones, tablet computers, desktop computers, notebooks, PDAs, robots, servers and other computing devices. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation of the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 7 may also include input and output devices, network access devices, buses, etc.

[0148] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0149] The memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. The memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 may also include both an internal storage unit of the electronic device 7 and an external storage device. The memory 71 is used to store the computer program and other programs and data required by the electronic device 7. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0150] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0151] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0153] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0157] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A face detection method, characterized in that: include: Acquire the target image to be detected; Performing face detection in the target image using a preset face detection model to obtain a face detection result of the target image; Among them, the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2.

2. The face detection method according to claim 1, characterized in that: The training process of the face detection model includes: Get the preset face detection training sample set; Dividing the face detection frames in the face detection training sample set into N detection frame groups corresponding to faces of different scales; wherein the nth detection frame group corresponds to the nth convolution kernel in the re-parameterization module, 1≤n≤N; In each training batch, determine the training loss for each of the N detection box groups. The face detection model is trained according to the training losses of the N detection frame groups to obtain a trained face detection model.

3. The face detection method according to claim 2, characterized in that: The training of the face detection model according to the training loss of the N detection frame groups includes: For the target convolution kernel, respectively determine the gradient of the training loss of the N detection frame groups; wherein the target convolution kernel is any convolution kernel in the reparameterization module; Determine a total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups; wherein the gradient weight of the first detection frame group is greater than the gradient weight of the second detection frame group, the first detection frame group is the detection frame group corresponding to the target convolution kernel, and the second detection frame group is the other detection frame group except the first detection frame group; According to the total gradient corresponding to the target convolution kernel, the model parameters of the target convolution kernel are adjusted.

4. The face detection method according to claim 3, characterized in that: The re-parameterization module includes a first convolution kernel, a second convolution kernel and a third convolution kernel, the face scale corresponding to the first convolution kernel is larger than the face scale corresponding to the second convolution kernel, and the face scale corresponding to the second convolution kernel is larger than the face scale corresponding to the third convolution kernel; Determining the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups includes: When the target convolution kernel is the first convolution kernel, determining a first weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a first gradient weight of the first detection frame group; wherein the first detection frame group is a detection frame group corresponding to the first convolution kernel; Determine a first weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the first gradient weight of the second detection frame group; wherein the second detection frame group is a detection frame group corresponding to the second convolution kernel; Determining a first weighted gradient of the third detection frame grouping according to a gradient of a training loss of the third detection frame grouping and a first gradient weight of the third detection frame grouping; wherein the third detection frame grouping is a detection frame grouping corresponding to the third convolution kernel; Determine a total gradient corresponding to the first convolution kernel according to the first weighted gradient of the first detection box group, the first weighted gradient of the second detection box group, and the first weighted gradient of the third detection box group.

5. The face detection method according to claim 4, characterized in that: The determining of the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups also includes: When the target convolution kernel is the second convolution kernel, determining a second weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a second gradient weight of the first detection frame group; Determine a second weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the second gradient weight of the second detection frame group; Determine a second weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the second gradient weight of the third detection frame group; Determine a total gradient corresponding to the second convolution kernel according to the second weighted gradient of the first detection box group, the second weighted gradient of the second detection box group, and the second weighted gradient of the third detection box group.

6. The face detection method according to claim 4, characterized in that: The determining of the total gradient corresponding to the target convolution kernel according to the gradient of the training loss of the N detection frame groups and the preset gradient weights of the N detection frame groups also includes: When the target convolution kernel is the third convolution kernel, determining a third weighted gradient of the first detection frame group according to a gradient of a training loss of the first detection frame group and a third gradient weight of the first detection frame group; Determine a third weighted gradient of the second detection frame group according to the gradient of the training loss of the second detection frame group and the third gradient weight of the second detection frame group; Determine a third weighted gradient of the third detection frame group according to the gradient of the training loss of the third detection frame group and the third gradient weight of the third detection frame group; Determine a total gradient corresponding to the third convolution kernel according to the third weighted gradient of the first detection frame group, the third weighted gradient of the second detection frame group, and the third weighted gradient of the third detection frame group.

7. The face detection method according to any one of claims 1 to 6, characterized in that: The convolution kernel corresponding to the minimum-scale face in the re-parameterization module is a central difference convolution kernel.

8. A face detection device, characterized in that: include: A target image acquisition module is used to acquire a target image to be detected; A face detection module, used to perform face detection in the target image using a preset face detection model to obtain a face detection result of the target image; Among them, the re-parameterization module of the face detection model during the training process includes N convolution kernels corresponding to faces of different scales, where N is an integer greater than or equal to 2.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the face detection method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the face detection method according to any one of claims 1 to 7 are implemented.

Images