A selective step-by-step lightweight face detection method based on feature enhancement

Abstract

The present invention provides a method and system for training a face detection model. The method includes: performing data augmentation on a training dataset to obtain an augmented image set; performing basic feature extraction on each image in the augmented image set to obtain an original feature map of a first branch; performing feature enhancement based on a validity mechanism on the obtained original feature map to obtain an enhanced feature map of a second branch; initializing training parameters; constructing a stepwise classification group and a stepwise regression group comprising the first branch and the second branch according to a selective strategy; and performing a weighted summation of the stepwise losses of the constructed stepwise classification group and stepwise regression group for supervised model learning until the model converges. Furthermore, the present invention provides a face detection method and system.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more particularly to a selective step-by-step lightweight face detection method based on feature enhancement. Background Art

[0002] The widespread use of intelligent terminal devices has promoted a shift in human thinking and redefined the nature of human social interaction. Object detection, the most widely used branch of computer vision, relies on algorithms to extract visual instances of specific objects (such as people, animals, plants, and cars) from digital images. Face detection replaces the arduous human visual processing with efficient machine aggregation and analysis, profoundly impacting the production and development of our modern society. Currently, face detection has achieved breakthroughs in privacy, security, and social governance, such as 3D facial recognition unlocking, suspect tracking, facial verification, and payment.

[0003] With the rapid development of deep learning, face detection methods based on convolutional neural networks (CNNs), with their powerful representation learning and nonlinear modeling capabilities, have gradually replaced traditional manual template matching methods. By autonomously learning facial features, they significantly improve detection accuracy. On recognized face detection benchmarks, leading-edge face detection methods typically have model sizes of hundreds of megabytes or more. These methods sacrifice lightweight models in favor of more complex models and more comprehensive parameters to achieve improved accuracy.

[0004] Applying face detection to edge and terminal devices (such as mobile phones, embedded systems, and even cameras) is becoming a leading demand among manufacturers. However, due to computing power and memory limitations, these devices are completely unable to support the hundreds of megabytes of model size required by some of the most advanced face detection methods. Furthermore, the performance differences between these methods on recognized face detection benchmarks are minimal and almost imperceptible in real-world applications. Therefore, R&D for face detection on edge and terminal devices should focus on improving real-time performance to achieve true industrial-grade implementation standards.

[0005] Real-time performance requires fast network model inference, which necessitates streamlined parameter calculations and model structure. This, to a certain extent, limits the adequacy of feature extraction and impacts the accuracy of detection results. Therefore, lightweight face detection methods with high real-time performance still have significant room for improvement in accuracy in practical applications. However, existing technologies are not yet adaptable to edge and terminal devices. Therefore, it is desirable to provide a face detection method and system that strikes a good balance between real-time performance and accuracy. Summary of the Invention

[0006] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] The present invention provides a selective step-by-step lightweight face detection method based on feature enhancement, focusing on the following three aspects for improvement and optimization: First, to ensure the real-time performance of the network model, a lightweight residual backbone network is built, and local residual connections are combined with global residual connections, striving to extract more sufficient and comprehensive feature information within a limited scale space. Second, focus on mining the current layer features, and realize the expansion of the single-branch architecture to the dual-branch architecture through a feature enhancement module based on the effectiveness mechanism, and accordingly design a step-by-step loss to adapt to the step-by-step learning capabilities of each branch and each level of feature map, thereby expanding the extraction mode of facial features. Third, a selective strategy is applied during reasoning, specifying the bottom layer and detail features to focus on classification scoring, and specifying high-level semantic features with strong discriminative power to focus on position regression, so as to avoid mutual interference between feature maps of different levels as much as possible, making the algorithm model more efficient.

[0008] According to one aspect of the present invention, a method for training a face detection model is provided, the method comprising:

[0009] Perform data augmentation on the training dataset to obtain an augmented image set;

[0010] Performing basic feature extraction on each image in the augmented image set to obtain an original feature map of the first branch;

[0011] Performing feature enhancement based on the validity mechanism on the obtained original feature map to obtain an enhanced feature map of the second branch;

[0012] Initialize training parameters;

[0013] constructing a stepwise classification group and a stepwise regression group including the first branch and the second branch according to a selective strategy; and

[0014] The weighted summation of the constructed step-by-step classification group and step-by-step regression group is used to supervise model learning until the model converges.

[0015] According to one embodiment of the present invention, performing data augmentation on the training data set to obtain an augmented picture set further includes:

[0016] Perform preliminary preprocessing on each image in the training dataset, including horizontal flipping and random cropping;

[0017] Data augmentation methods including color jittering and noise perturbation are used to simulate interference in unconstrained scenes to further process each image to obtain an augmented image set.

[0018] According to a further embodiment of the present invention, performing basic feature extraction on each picture in the augmented picture set to obtain an original feature map of the first branch further includes:

[0019] A lightweight residual backbone network is used to extract basic features of each image in the augmented image set to obtain an original feature map, wherein the lightweight residual backbone network adopts multiple Xception modules, and the Xception module linearly stacks convolutional layers through residual connections to achieve a depth-separable structure, wherein local residual connections are implemented inside the Xception module and global residual connections are implemented outside the lightweight residual backbone network.

[0020] According to a further embodiment of the present invention, performing feature enhancement based on a validity mechanism on the obtained original feature map to obtain an enhanced feature map of the second branch further includes:

[0021] Use a convolution kernel of size 1×1 to normalize the feature map of the current layer;

[0022] Perform transposed convolution on the upper layer feature map after the dilated convolution and the current layer feature map;

[0023] Feed the transposed feature map into the spatially separable convolutional network;

[0024] Processing the feature map after spatially separable convolution by mean pooling and maximum pooling respectively to obtain mean pooled and maximum pooled feature maps; and

[0025] The mean-pooled and maximum-pooled feature maps are passed into a shared multi-layer perceptron to obtain an enhanced feature map.

[0026] According to a further embodiment of the present invention, the selective strategy further comprises:

[0027] The original feature map of the lower level of the first branch and the enhanced feature map of the lower level of the second branch are constructed as a step classification group, and the original feature map of the higher level of the first branch and the enhanced feature map of the higher level of the second branch are constructed as a step regression group.

[0028] According to a further embodiment of the present invention, the constructed step-by-step loss of the step-by-step classification group and the step-by-step regression group is obtained by weighted summing the dual-branch loss of the step-by-step classification group and the dual-branch loss of the step-by-step regression group, wherein the dual-branch loss of the step-by-step classification group is determined based on the softmax loss function, and the dual-branch loss of the step-by-step regression group is determined based on the smooth L1 loss function.

[0029] According to another aspect of the present invention, a face detection method based on feature enhancement is provided, the method comprising:

[0030] Get sample images for face detection;

[0031] Inputting the acquired image into the face detection model trained using the aforementioned method; and

[0032] The face detection model processes and marks the detected faces with rectangular frames in the acquired images.

[0033] According to another aspect of the present invention, a system for training a face detection model is provided, wherein the system comprises:

[0034] A data augmentation module, configured to perform data augmentation on the training dataset to obtain an augmented picture set;

[0035] a feature extraction module, configured to perform basic feature extraction on each image in the augmented image set to obtain an original feature map of a first branch;

[0036] a feature enhancement module, configured to perform feature enhancement based on a validity mechanism on the obtained original feature map to obtain an enhanced feature map of a second branch;

[0037] a selective construction module configured to initialize training parameters and construct a stepwise classification group and a stepwise regression group including the first branch and the second branch according to a selective strategy; and

[0038] A loss calculation module is configured to perform a weighted summation of the stepwise losses of the constructed stepwise classification group and the stepwise regression group for supervised model learning until the model converges.

[0039] According to one embodiment of the present invention, the feature enhancement module is further configured to:

[0040] Use a convolution kernel of size 1×1 to normalize the feature map of the current layer;

[0041] Perform transposed convolution on the upper layer feature map after the dilated convolution and the current layer feature map;

[0042] Feed the transposed feature map into the spatially separable convolutional network;

[0043] Processing the feature map after spatially separable convolution by mean pooling and maximum pooling respectively to obtain mean pooled and maximum pooled feature maps; and

[0044] The mean-pooled and maximum-pooled feature maps are passed into a shared multi-layer perceptron to obtain an enhanced feature map.

[0045] According to another aspect of the present invention, a face detection system based on feature enhancement is provided, characterized in that the system comprises:

[0046] A memory storing a face detection model trained using the method of the aforementioned aspects and computer executable instructions; and

[0047] At least one processor, wherein the computer executable instructions, when executed, may cause the at least one processor to perform the following operations:

[0048] Get sample images for face detection;

[0049] Inputting the acquired picture into the face detection model; and

[0050] The face detection model processes and marks the detected faces with rectangular frames in the acquired images.

[0051] Compared with the solutions in the prior art, the selective step-by-step lightweight face detection method and system based on feature enhancement provided by the present invention has at least the following advantages:

[0052] (1) The present invention makes up for the neglect of real-time performance in existing methods by constructing a lightweight residual backbone network and combining local residual connections with global residual connections to maintain sufficiency and comprehensiveness as much as possible when extracting features within a limited scale space;

[0053] (2) Considering that existing methods have neglected the attention to the current feature layer, the present invention not only focuses on capturing contextual clues, but also fully collects the feature map information of the current layer. It adopts a feature enhancement module based on the effectiveness mechanism to construct a dual-branch architecture to accurately extract useful information, and accordingly designs a step-wise loss to match the step-wise learning ability of the feature maps at each level.

[0054] (3) Compared with the existing methods, the present invention further reduces the interference between feature maps of different levels. It applies a selective strategy during reasoning, selects low-level detail features to focus on classification scoring, selects high-level semantic features with strong discriminative power to participate in position regression, and coordinates the processing of feature information at all levels to improve the efficiency of the algorithm model. To a certain extent, it can alleviate the problem of accuracy degradation caused by putting all the interfering feature maps of each level into classification and regression calculation in the existing methods, and achieve good gains.

[0055] (4) The present invention can basically meet the real-time requirements of industrial grade, has low requirements for computing power and memory, and can be deployed on edge and terminal devices. In addition, the present invention can be trained and tested from scratch without the need for pre-trained networks; and

[0056] (5) The present invention can maintain a high detection accuracy rate when detecting faces with medium and low detection difficulty in unconstrained scenes, such as faces with different scales, blur, light intensity, posture, facial occlusion, and makeup, and has extremely high real-time and comprehensiveness.

[0057] These and other features and advantages will become apparent from reading the following detailed description and referring to the associated drawings.It is to be understood that both the foregoing general description and the following detailed description are illustrative only and are not restrictive of the aspects claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order that the manner in which the above-mentioned features of the present invention are understood in detail, a more particular description of the contents briefly summarized above may be given with reference to various embodiments, some aspects of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only certain typical aspects of the invention and are not to be considered limiting of its scope, as the description may admit to other equally effective aspects.

[0059] Figure 1 An example architecture diagram of a system for training a selective step-by-step lightweight face detection model based on feature enhancement according to an embodiment of the present invention is shown.

[0060] Figure 2 A flowchart of a method for training a selective step-by-step lightweight face detection model based on feature enhancement according to an embodiment of the present invention is shown.

[0061] Figure 3 A network structure diagram of a selective step-by-step lightweight face detection model based on feature enhancement according to an embodiment of the present invention is shown.

[0062] Figure 4 A schematic diagram of a facial image processing and enhancement method according to an embodiment of the present invention is shown.

[0063] Figure 5 A schematic diagram of an Xception module according to an embodiment of the present invention is shown.

[0064] Figure 6 A structural diagram of a lightweight residual backbone network according to an embodiment of the present invention is shown.

[0065] Figure 7 A structural diagram of a feature enhancement module based on a validity mechanism according to an embodiment of the present invention is shown.

[0066] Figure 8 A schematic diagram of constructing a stepwise classification group and a stepwise regression group based on a selective strategy according to an embodiment of the present invention is shown.

[0067] Figure 9 A diagram shows the effect of detecting face samples on the WIDERFACE test set using a trained face detection model according to an embodiment of the present invention.

[0068] Figure 10 The figure shows the detection accuracy of the trained face detection model according to one embodiment of the present invention on the Easy, Medium, and Hard validation sets of WIDER FACE. DETAILED DESCRIPTION

[0069] The present invention will be described in detail below with reference to the accompanying drawings, and the features of the present invention will be further apparent in the following specific description.

[0070] Figure 1 FIG. 1 is an exemplary architecture diagram of a system 100 for training a selective step-by-step lightweight face detection model based on feature enhancement according to an embodiment of the present invention. Figure 1 As shown in , the system 100 of the present invention includes: a data augmentation module 101, a feature extraction module 102, a feature enhancement module 103, a selective construction module 104 and a loss calculation module 105.

[0071] The data augmentation module 101 can be configured to perform data augmentation on the face detection benchmark training set to obtain an augmented picture set. The training set can be, for example, WIDER FACE, which is currently the most authoritative face detection benchmark training set. Specifically, the data augmentation module 101 can horizontally flip and randomly crop each picture in the training set as preliminary preprocessing. As an example, the data augmentation module 101 can first expand each picture in the input picture set to 4 times its original size, then mirror-flip each picture horizontally, and finally randomly crop out a 640×640 area. Furthermore, the data augmentation module 101 can use color jitter, noise perturbation, etc. to simulate interference in unconstrained scenes, and then enhance the obtained preliminary preprocessing results to varying degrees, thereby obtaining an augmented picture set after comprehensive processing.

[0072] The feature extraction module 102 can be configured to use a lightweight residual backbone network to extract basic features from the augmented image set to obtain the original feature map for final prediction. The lightweight residual backbone network can adopt several Xception modules, wherein the Xception module is a structure that linearly stacks convolution layers through residual connections to achieve depth-separable, which includes several convolution branches with different convolution kernel sizes. In addition, the activation function can use a series rectified linear unit to implement local residual connections inside the Xception module and global residual connections outside the residual backbone network. In the lightweight residual backbone network, Xception1, Xception2, Xception3, Xception4, conv5_4, and conv6_4 can be selected for the final prediction, and the original feature map sizes are 156×156, 78×78, 39×39, 19×19, 9×9, and 4×4, respectively.

[0073] The feature enhancement module 103 can be configured to apply an effectiveness mechanism to integrate information of different dimensions, enhance the original feature map obtained to obtain an enhanced feature map, wherein the effectiveness mechanism can ensure that the input information focused by the channel is meaningful, refine the spatial information of the feature map, and locate the effective features in a more accurate manner. More specifically, in some cases, the feature enhancement module 103 can use a convolution kernel of size 1×1 to normalize the current feature map, transpose convolution the upper feature map after the hole convolution and the current feature map, and then send the transposed feature map to the spatially separable convolution network, whose dimension is consistent with the dimension of the initial feature map, and then process the feature map after the spatially separable convolution by mean pooling and maximum pooling respectively to aggregate more sufficient spatial information, and finally pass the mean pooling features and maximum pooling features into the shared multi-layer perceptron to obtain the enhanced feature map.

[0074] The selective construction module 104 can initialize the training parameters and construct a step classification group and a step regression group according to the selective strategy, which are responsible for category scoring and position regression respectively. In some cases, the selective construction module 104 can select Xception1, Xception2 and Xception3 in the lightweight residual backbone network as the step classification group and conv5_4 and conv6_4 as the step regression group for final prediction, wherein for the step classification group, the Xception1, Xception2, and Xception3 in the original lightweight residual backbone network first filter a large number of easily distinguishable negative samples according to a preset intersection-over-union ratio threshold, and for the step regression group, the Xception4, conv5_4, and conv6_4 in the original lightweight residual backbone network first roughly adjust the anchor point position.

[0075] The loss calculation module 105 can be configured to use step-by-step loss to guide and supervise the autonomous learning process of the model based on the step-by-step learning capabilities of the feature maps of each branch and each level, wherein the step-by-step loss is obtained by weighted summing the dual-branch loss of the step-by-step classification group and the dual-branch loss of the step-by-step regression group. When the calculated weighted loss no longer increases but stabilizes in a smaller value range, training is stopped to obtain a trained model for face detection.

[0076] Those skilled in the art will appreciate that the system and its modules of the present invention can be implemented in either hardware or software, and the modules can be combined or integrated in any suitable manner. Furthermore, the system can be implemented on edge or terminal devices for real-time training of face detection models for face detection.

[0077] Figure 2 A flowchart of a method 200 for training a selective step-wise lightweight face detection model based on feature enhancement according to an embodiment of the present invention is shown. The method 200 comprises the following steps:

[0078] Step S1: perform data augmentation on the face detection benchmark training set (e.g., the WIDER FACE training set), which mainly includes the following two aspects:

[0079] Step S1.1: Perform horizontal flipping and random cropping on each image in the training set as preliminary preprocessing, such as Figure 4 As shown in the figure, the specific operation is: first, each input image is expanded to 4 times the original size, then each image is mirrored and flipped horizontally, and finally a 640×640 area is randomly cropped, that is, the following formula is applied for processing:

[0080] x preprocess =Crop(Flip(Extend(x inputFormula 1

[0081] In the above formula 1, x input Represents the input training set image. The Extend operation expands the image by filling the mean. The Flip operation represents a random horizontal flip. Crop is a random cropping operation. preprocess It represents the corresponding preliminary preprocessing result, and its size is unified to 640×640.

[0082] Step S1.2: Use data enhancement to simulate interference in unconstrained scenes, where data enhancement methods may include color dithering and noise perturbation methods. Specifically, color dithering refers to adjusting the saturation, brightness, contrast, and sharpness of the input image according to arbitrarily generated random factors, taking into account the differences in light intensity, background atmosphere, shooting conditions, etc. Noise perturbation mainly involves the addition of Gaussian white noise and salt and pepper noise, where Gaussian noise is an additive noise, which means that the noise amplitude obeys the Gaussian distribution, that is, the number of noise points of a certain intensity is the largest, and the farther away from this intensity, the fewer the noise points. Salt and pepper noise is an impulse noise, which can produce black and white light and dark point noise on the original image by randomly changing the values ​​of certain pixels. To put it more vividly, it is like sprinkling pepper and salt on the image, which is a logical noise. In summary, the preliminary preprocessing result x obtained in step S1.1 is again processed. preprocess Perform different degrees of enhancement to obtain the comprehensive processed augmented image x process , as shown below:

[0083]

[0084] In formula 2, the Color operation represents the color dithering method, and the Noise (Gaussian) and Noise (Salt & Pepper) operations represent the image plus Gaussian noise and salt and pepper noise respectively. Figure 4 As shown, the fifth row performs color dithering enhancement on the image cropped from the fourth row. The sixth and seventh rows add different degrees of Gaussian noise and salt and pepper noise to the image cropped from the fourth row, respectively, to enhance the detection stability of the model for any environmental external factors.

[0085] Step S2, based on the augmented image obtained in step S1, uses a lightweight residual backbone network to extract basic features, and adopts a feature enhancement module based on the validity mechanism to implement a dual-branch architecture to further extract enhanced features. It mainly includes the following two aspects:

[0086] Step S2.1: Use a lightweight residual backbone network to extract basic features from the augmented input image to obtain the original feature map of the first branch. The lightweight residual backbone network can use the Xception module, which is a structure that linearly stacks convolutional layers through residual connections to achieve depth-separable. Its principle is as follows Figure 5 As shown. Figure 5 As explained in [1], the Xception module is composed of multiple convolution branches with different convolution kernel sizes. These branches can provide a richer and more comprehensive receptive field. The activation function uses a series rectified linear unit to reduce the number of output channels. In addition, local residual connections are implemented inside the Xception module, and global residual connections are implemented outside the backbone network. The fusion of the two can not only accelerate the convergence of the network model, but also make more full use of the underlying original features to supplement and restore the target's detailed information. The overall architecture of the lightweight residual backbone network and its related parameter settings are shown in the figure. Figure 6 As shown in the figure, Xception1, Xception2, Xception3, Xception4, conv5_4, and conv6_4 are selected for the final prediction, and the sizes of the original feature maps of the first branch are 156×156, 78×78, 39×39, 19×19, 9×9, and 4×4, respectively.

[0087] Step S2.2: Use the feature enhancement module to implement the dual-branch architecture, apply the validity mechanism to integrate information of different dimensions, and enhance the original feature map of the first branch used for prediction in step S2.1 to obtain an enhanced second branch corresponding to the first branch. The validity mechanism can ensure that the input information focused by the channel is meaningful, refine the spatial information of the original feature map, and locate the effective features in a more accurate way. The subunit in the original feature map of the upper layer is denoted as u (i,j,l) , the subunit in the original feature map of the current layer is nu (i-ε,j-ε,l) 、nu (i-ε,j,l) 、……、nu (i,j+ε,l) 、nu (i+ε,j+ε,l) , then the subunit su of the enhanced feature map (i,j,l) Expressed as:

[0088]

[0089] In formula 3, u (i,j,l)It is the subunit mapped by the coordinates (i, j) in the feature map of the lth layer, f points to a series of maximum pooling operations (Max-pooling), average pooling operations (Average-pooling), spatially separable convolution operations (Separable Convolution), transposed convolution operations (Transposed Convolution) and dilation convolution operations (Dilation Convolution), σ represents the Sigmoid activation function, η is the weighting parameter, and its value is obtained after training convergence. MLP (Multilayer Perceptron) is a multi-layer perceptron, and its layers are connected in a fully connected manner. The bottom layer, middle layer, and top layer are the input layer, hidden layer, and output layer respectively.

[0090] The specific structure of the feature enhancement module based on the effectiveness mechanism is as follows Figure 7 As shown, the specific implementation is as follows:

[0091] (1) Use a convolution kernel of size 1×1 to normalize the feature map of the current layer;

[0092] (2) Perform transposed convolution on the upper layer feature map after the dilated convolution and the current layer feature map;

[0093] (3) The transposed feature map is then fed into a spatially separable convolutional network, with its dimension remaining consistent with that of the initial feature map.

[0094] (4) Process the feature map after spatially separable convolution using mean pooling and maximum pooling to aggregate more sufficient spatial information;

[0095] (5) Finally, the mean pooling feature and the maximum pooling feature are passed to the shared multi-layer perceptron. In order to reduce the parameter overhead, Figure 7 A multi-layer perceptron with only one hidden layer is used, and the Sigmoid function is selected as the activation function.

[0096] In step S3, after the training parameters are initialized, a selective strategy is applied to perform a weighted summation of the step-wise losses of the constructed step-wise classification group and the step-wise regression group to guide and supervise the autonomous learning process of the model. After the model converges, it can be saved for face detection.

[0097] Step S3.1: Initialize the training parameters. The specific settings are shown in Table 1 below.

[0098] Table 1 Training parameter settings

[0099]

[0100] The optimizer uses the stochastic gradient descent (SGD) method with a momentum value of 0.87. In addition, to prevent overfitting, the weight decay value is set to 10 -6 It is important to note that, considering the continuous deepening of the network learning process, the learning rate can be set as follows: as the number of iterations increases, when the number of iterations is within the set step list {30000, 60000, 90000}, the learning rate is reduced to the original 0.1. This can prevent the network parameters from missing the optimal value due to excessive learning rate when approaching the global optimal solution. In addition, since there is no need to pre-train the network, training can be started from scratch. The network's initial weights can be randomly generated using, for example, the Xavier method, where Xavier is a commonly used parameter initialization method for deep learning.

[0101] Step S3.2: During training, according to the selective strategy, a step classification group and a step regression group are constructed, which are responsible for category scoring and position regression tasks respectively. The step classification group may include the low-level feature map of the first branch and the low-level feature map of the second branch, while the step regression group may include the high-level feature map of the first branch and the high-level feature map of the second branch. Figure 8 FIG. 1 shows a schematic diagram of constructing a stepwise classification group and a stepwise regression group based on a selective strategy according to an embodiment of the present invention. Figure 8As shown, the selective strategy involves selecting Xception1, Xception2, Xception3, Xception4, conv5_4, and conv6_4 generated by the lightweight residual backbone network in step S2.1 for final prediction. Feature maps with an "*" appended to their names indicate the corresponding feature maps processed by the validity-based feature enhancement module in step S2.2. Regarding the step classification group, Xception1, Xception2, and Xception3 in the original lightweight residual backbone network first filter out most easily distinguishable negative samples based on a preset intersection-over-union threshold. This eliminates unnecessary search computational costs for Xception1*, Xception2*, and Xception3*, and to some extent reduces the risk of increased false positive rates due to an imbalanced distribution of positive and negative samples. For higher-level feature maps, constructing step classification groups is unnecessary, primarily to simplify model parameters. In the case of the stepwise regression group, the original lightweight residual backbone network, Xception4, conv5_4, and conv6_4, first roughly adjust the anchor point positions to provide better initialization for Xception4*, conv5_4*, and conv6_4*. This is a coarse-to-fine process. Preferably, regression calculations are performed only on high-level features, as low-level features themselves are inherently weak in robustness and expressiveness. Blindly incorporating them into regression calculations can lead to more misjudgments, thus decreasing detection accuracy rather than improving it.

[0102] Step 3.3: Based on the step-wise learning capabilities of the feature maps of each branch and each level, a step-wise loss is used to guide and supervise the autonomous learning process of the model. The step-wise loss is the weighted sum of the two-branch loss of the step-wise classification group and the two-branch loss of the step-wise regression group. The detailed calculation process is as follows:

[0103] (1) The basic category scoring is trained by softmax loss, which is expressed as:

[0104]

[0105] In formula 4, x k represents the actual category label, z m represents the input of the softmax layer, f(z m ) represents the output predicted by the softmax layer, and T is the number of categories in the training dataset.

[0106] In addition, the basic position regression is trained by the smooth L1 loss, which is expressed as:

[0107]

[0108] In formula 5, y (i) represents the actual location label, Represents the coordinate label information predicted by the network model, and Ω represents the set of regions where the prior box is a positive sample.

[0109] (2) The dual-branch loss of progressive classification group (DLPCG) is defined as follows:

[0110]

[0111] In formula 6, N cls 、N c ' ls They represent the number of positive anchor boxes used for category scoring in the first branch of the lightweight residual backbone network in the step classification group and the second branch of the feature enhancement module based on the validity mechanism, i is the index of the anchor point, L cls It represents the softmax loss of the two categories of face and background, p i ,q i denote the i-th prediction anchor point in the first branch of the lightweight residual backbone network and its second branch after the feature enhancement module based on the validity mechanism, is the corresponding true category label, λ is the weighting coefficient, and in the present invention, λ is set to 0.5 to match the compensation of the anchor point scale.

[0112] (3) The dual-branch loss of progressive regression group (DLPRG) is defined as follows:

[0113]

[0114] In formula 7, N reg 、N r ' eg They represent the number of positive anchor boxes for position regression in the first branch of the lightweight residual backbone network in the stepwise regression group and the second branch of the feature enhancement module based on the validity mechanism, i is the index of the anchor point, It means that in the first branch of the lightweight residual backbone network, the anchor point a is used i During detection, the predicted box t i With the true value box The parameterized smooth L1 loss between It means that in the second branch of the feature enhancement module based on the validity mechanism, the anchor point sa is used iDuring detection, the predicted box x i With the true value box The parameterized smooth L1 loss between λ and λ is a weighting coefficient. In the present invention, λ is set to 0.5 to match the compensation of the anchor point scale.

[0115] (4) The weighted sum of the two-branch loss of the step classification group and the two-branch loss of the step regression group is obtained, and the total loss function is as follows:

[0116] L=αL DLPCG +βL DLPRG Formula 8

[0117] In Equation 8, α and β are hyperparameters that balance the above two. For example, considering that the number of positive anchor boxes in the stepwise regression group is approximately three times that in the stepwise classification group, α and β can be set to 1 and 3 respectively.

[0118] When the calculated dual-branch weighted loss no longer increases but stabilizes in a smaller value range (e.g., (0, 0.5]), the trained face detection model can be obtained for face detection. Otherwise, step S3 is repeated. The network structure diagram of the trained face detection model is shown in FIG. Figure 3 As shown, the lightweight residual backbone network can use Xception1, Xception2, Xception3, Xception4, conv5_4, and conv6_4 to extract basic features to obtain the original feature map of the first branch. The original feature map is processed by a feature enhancement module based on a validity mechanism to obtain the corresponding enhanced feature map of the second branch, wherein local residual connections are used inside the Xception module and global residual connections are used outside the backbone network.

[0119] When the model is put into actual detection, only the enhanced branch output can be used as a reference. The trained model can be used to detect some face samples in the WIDER FACE test set that have attributes such as different scales, blur, light intensity, posture, face occlusion, and makeup, and the faces can be marked with rectangular boxes, such as Figure 9 As shown in Figure 2, high detection accuracy is achieved, especially in unconstrained scenarios of medium and low difficulty. The accuracy of the present invention on the Easy, Medium, and Hard validation sets of the public WIDER FACE is 89.7%, 87.5%, and 50.9%, respectively. Figure 10 As shown, the proposed method achieves good gains within the scope of lightweight networks. It can detect up to 148 images per second on a GPU (graphics processing unit) platform, and 117 images per second using only a CPU (central processing unit), greatly satisfying the real-time requirements of face detection tasks.

[0120] What has been described above includes examples of various aspects of the claimed subject matter. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the claimed subject matter, but one skilled in the art will recognize that many further combinations and permutations of the claimed subject matter are possible. Thus, the disclosed subject matter is intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

Claims

1. A method for training a face detection model, characterized in that: The method comprises: Perform data augmentation on the training dataset to obtain an augmented image set; Using a lightweight residual backbone network to extract basic features from each image in the augmented image set to obtain an original feature map of the first branch, wherein the lightweight residual backbone network adopts multiple Xception modules, and the Xception modules are linearly stacked with convolutional layers through residual connections to achieve a depth-wise separable structure, wherein local residual connections are implemented within the Xception modules and global residual connections are implemented outside the lightweight residual backbone network; Performing feature enhancement based on a validity mechanism on the obtained original feature map to obtain an enhanced feature map of a second branch, wherein the feature enhancement based on the validity mechanism further includes: Use a convolution kernel of size 1×1 to normalize the feature map of the current layer; Perform transposed convolution on the upper layer feature map after the dilated convolution and the current layer feature map; Feed the transposed feature map into the spatially separable convolutional network; Processing the feature map after spatially separable convolution by mean pooling and maximum pooling respectively to obtain mean pooled and maximum pooled feature maps; and Passing the mean-pooled and maximum-pooled feature maps into a shared multi-layer perceptron to obtain an enhanced feature map; Initialize training parameters; constructing a stepwise classification group and a stepwise regression group including the first branch and the second branch according to a selective strategy; and The weighted summation of the constructed step-by-step classification group and step-by-step regression group is used to supervise model learning until the model converges.

2. The method according to claim 1, wherein Performing data augmentation on the training dataset to obtain an augmented image set further includes: Perform preliminary preprocessing on each image in the training dataset, including horizontal flipping and random cropping; Data augmentation methods including color jittering and noise perturbation are used to simulate interference in unconstrained scenes to further process each image to obtain an augmented image set.

3. The method according to claim 1, wherein The selective strategy further comprises: The original feature map of the lower level of the first branch and the enhanced feature map of the lower level of the second branch are constructed as a step classification group, and the original feature map of the higher level of the first branch and the enhanced feature map of the higher level of the second branch are constructed as a step regression group.

4. The method according to claim 1, wherein The constructed step-wise loss of the step-wise classification group and the step-wise regression group is obtained by weighted summing of the dual-branch loss of the step-wise classification group and the dual-branch loss of the step-wise regression group, wherein the dual-branch loss of the step-wise classification group is determined based on the softmax loss function, and the dual-branch loss of the step-wise regression group is determined based on the smooth L1 loss function.

5. A face detection method based on feature enhancement, characterized in that: The method comprises: Get sample images for face detection; Inputting the acquired image into a face detection model trained using the method according to any one of claims 1 to 4; and The face detection model processes and marks the detected faces with rectangular frames in the acquired images.

6. A system for training a face detection model, characterized in that: The system comprises: A data augmentation module, configured to perform data augmentation on the training dataset to obtain an augmented picture set; a feature extraction module configured to perform basic feature extraction on each image in the augmented image set using a lightweight residual backbone network to obtain an original feature map of a first branch, wherein the lightweight residual backbone network adopts multiple Xception modules, and the Xception modules linearly stack convolutional layers using residual connections to achieve a depth-wise separable structure, wherein local residual connections are implemented within the Xception modules and global residual connections are implemented outside the lightweight residual backbone network; A feature enhancement module is configured to perform feature enhancement based on a validity mechanism on the obtained original feature map to obtain an enhanced feature map of the second branch, wherein the feature enhancement based on the validity mechanism further includes: Use a convolution kernel of size 1×1 to normalize the feature map of the current layer; Perform transposed convolution on the upper layer feature map after the dilated convolution and the current layer feature map; Feed the transposed feature map into the spatially separable convolutional network; Processing the feature map after spatially separable convolution by mean pooling and maximum pooling respectively to obtain mean pooled and maximum pooled feature maps; and Passing the mean-pooled and maximum-pooled feature maps into a shared multi-layer perceptron to obtain an enhanced feature map; a selective construction module configured to initialize training parameters and construct a stepwise classification group and a stepwise regression group including the first branch and the second branch according to a selective strategy; and A loss calculation module is configured to perform a weighted summation of the stepwise losses of the constructed stepwise classification group and the stepwise regression group for supervised model learning until the model converges.

7. A face detection system based on feature enhancement, characterized in that: The system comprises: A memory storing a face detection model trained by the method according to any one of claims 1 to 4 and computer-executable instructions; and At least one processor, wherein the computer executable instructions, when executed, may cause the at least one processor to perform the following operations: Get sample images for face detection; Inputting the acquired picture into the face detection model; and The face detection model processes and marks the detected faces with rectangular frames in the acquired images.

Images