A face detection model training method, face detection method and related device

Abstract

This application discloses a face detection model training method, a face detection method, and related devices. A face attribute network branch is added after the first convolutional layer of the P-Net in the MTCNN network, and an attribute fusion layer is added between the second and third convolutional layers of the P-Net to construct a new P-Net. The new P-Net, the R-Net, and the O-Net in the MTCNN network are trained in stages using a face detection dataset to obtain trained new P-Net, R-Net, and O-Net. The label information of the face detection dataset includes face information and face attribute information. The trained new P-Net, R-Net, and O-Net are combined to construct a face detection model. This application improves the technical problem of the existing technology using the MTCNN network for face detection, which results in low face detection accuracy and affects face detection speed due to the simple structure of the P-Net.

Description

Technical Field

[0001] The present application relates to the field of face detection technology, and in particular to a face detection model training method, a face detection method and related devices. Background Art

[0002] With the development of the intelligent society, facial recognition technology has been applied in many scenarios, such as residential access control systems, train station entrance gates, and exhibition security systems. Facial recognition technology not only provides convenience to people but also ensures their safety. For example, in public places like train stations, using facial recognition to pass through gates can greatly improve traffic efficiency compared to traditional manual methods.

[0003] Face detection determines whether a face exists in a given image and, if so, outputs the face's location within the image. Face detection is the foundation of face recognition technology. Because a scene image captured by a camera may contain multiple faces and vary in size, the actual face recognition process requires pre-capturing facial images and then performing liveness detection on them to prevent cheating. Furthermore, face image quality assessments, such as blur detection and occlusion detection, are required. Only facial images that meet these requirements are further processed for facial feature extraction and face recognition. Therefore, the quality of the face detection model directly impacts the performance of subsequent tasks.

[0004] In practical applications, it's crucial not only to consider the accuracy of the face detection model but also whether the detection speed meets the current application scenario. It's well known that the deeper and more complex the deep learning model, and the greater the number of weights, the stronger its detection capabilities. However, while large models offer excellent face detection capabilities, they are difficult to deploy on resource-constrained mobile devices. Therefore, while improving the model's detection accuracy, it's also important to consider the model's speed and computing resource consumption.

[0005] In the existing technology, a lightweight network called MTCNN (Multi-task convolutional neural network) is commonly used for face detection. MTCNN consists of a cascade of three networks: P-Net, R-Net, and O-Net. Due to the simple structure of P-Net, the accuracy of face detection is low, resulting in a large number of generated face candidate windows. The face candidate windows generated by P-Net are then fed into the more complex R-Net and O-Net for further detection and recognition, which increases the computational complexity of the face detection model and affects the face detection speed. Summary of the Invention

[0006] The present application provides a face detection model training method, a face detection method and related devices, which are used to improve the existing technology of using the MTCNN network for face detection. Due to the simple structure of P-Net, the accuracy of face detection is low and the speed of face detection is affected.

[0007] In view of this, the first aspect of the present application provides a face detection model training method, comprising:

[0008] Adding a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and adding an attribute fusion layer between the second and third convolutional layers of the P-Net to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output of the third convolutional layer is connected to the input of the face prediction layer;

[0009] Training the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net, wherein the label information of the training samples in the face detection dataset includes face information and face attribute information;

[0010] The face detection model is constructed by combining the trained new P-Net, the trained R-Net and the trained O-Net.

[0011] Optionally, the training process of the new P-Net is:

[0012] Inputting the training samples in the face detection dataset into the new P-Net, performing a first convolution process on the training samples through the first convolution layer in the new P-Net, and obtaining a first facial feature map;

[0013] Performing a second convolution process on the first facial feature map through the second convolution layer in the new P-Net to obtain a second facial feature map;

[0014] Performing a third convolution process on the first facial feature map through the fourth convolution layer in the new P-Net to obtain a facial attribute feature map;

[0015] Performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature, and sequentially performing a fourth convolution process and face prediction on the fused feature through the third convolution layer and the face prediction layer to obtain a face prediction result corresponding to the training sample;

[0016] Performing pooling processing and facial attribute prediction on the facial attribute feature map in sequence through the global average pooling layer and the attribute prediction layer to obtain facial attribute prediction results corresponding to the training sample;

[0017] A face loss value is calculated based on the face prediction result of the training sample and the face information in the label information, an attribute loss value is calculated based on the face attribute prediction result of the training sample and the face attribute information in the label information, and the network parameters of the new P-Net are updated by the face loss value and the attribute loss value until the new P-Net converges.

[0018] Optionally, the new P-Net further includes a maximum pooling layer, wherein the input end of the maximum pooling layer is connected to the output end of the fourth convolutional layer, and the output end is connected to the input end of the attribute information fusion layer;

[0019] Correspondingly, the method further includes: performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature;

[0020] Performing a maximum pooling operation on the facial attribute feature map through the maximum pooling layer to obtain a facial attribute weight map;

[0021] The step of fusing the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature includes:

[0022] The attribute information fusion layer performs a point-by-point multiplication operation on the second facial feature map and the facial attribute weight map to obtain a fused feature.

[0023] A second aspect of the present application provides a face detection method, comprising:

[0024] Get the face image to be detected;

[0025] The image to be detected is input into a face detection model for face detection to obtain a face detection result of the image to be detected, wherein the face detection model is obtained by any face detection model training method described in the first aspect.

[0026] A third aspect of the present application provides a face detection model training device, comprising:

[0027] A first construction unit is configured to add a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and to add an attribute fusion layer between the second and third convolutional layers of the P-Net to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output end of the third convolutional layer is connected to the input end of the face prediction layer;

[0028] a training unit, configured to train the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net, wherein the label information of the training samples in the face detection dataset includes face information and face attribute information;

[0029] The second construction unit is used to combine the trained new P-Net, the trained R-Net and the trained O-Net to construct a face detection model.

[0030] Optionally, the training process of the new P-Net is:

[0031] Inputting the training samples in the face detection dataset into the new P-Net, performing a first convolution process on the training samples through the first convolution layer in the new P-Net, and obtaining a first facial feature map;

[0032] Performing a second convolution process on the first facial feature map through the second convolution layer in the new P-Net to obtain a second facial feature map;

[0033] Performing a third convolution process on the first facial feature map through the fourth convolution layer in the new P-Net to obtain a facial attribute feature map;

[0034] Performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature, and sequentially performing a fourth convolution process and face prediction on the fused feature through the third convolution layer and the face prediction layer to obtain a face prediction result corresponding to the training sample;

[0035] Performing pooling processing and facial attribute prediction on the facial attribute feature map in sequence through the global average pooling layer and the attribute prediction layer to obtain facial attribute prediction results corresponding to the training sample;

[0036] A face loss value is calculated based on the face prediction result of the training sample and the face information in the label information, an attribute loss value is calculated based on the face attribute prediction result of the training sample and the face attribute information in the label information, and the network parameters of the new P-Net are updated by the face loss value and the attribute loss value until the new P-Net converges.

[0037] A fourth aspect of the present application provides a face detection device, comprising:

[0038] An acquisition unit, used for acquiring a face image to be detected;

[0039] A detection unit is used to input the image to be detected into a face detection model for face detection to obtain a face detection result of the image to be detected, wherein the face detection model is obtained by the face detection model training method described in any one of the first aspects.

[0040] A fifth aspect of the present application provides an electronic device, comprising: a processor and a memory;

[0041] The memory is used to store program code and transmit the program code to the processor;

[0042] The processor is used to execute any one of the face detection model training methods described in the first aspect, or execute the face detection method described in the second aspect according to the instructions in the program code.

[0043] In a sixth aspect, the present application provides a computer-readable storage medium for storing program code, wherein the program code is used to execute any one of the face detection model training methods described in the first aspect, or to execute the face detection method described in the second aspect.

[0044] It can be seen from the above technical solutions that this application has the following advantages:

[0045] The present application provides a face detection model training method, comprising: adding a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and adding an attribute fusion layer between the second convolutional layer and the third convolutional layer of the P-Net to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output end of the third convolutional layer is connected to the input end of the face prediction layer; training the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net, wherein the label information of the training samples in the face detection dataset includes face information and face attribute information; and constructing a face detection model by combining the trained new P-Net, the trained R-Net, and the trained O-Net.

[0046] In this application, the P-Net in the original MTCNN network is improved by adding a face attribute network branch after the first convolutional layer of the P-Net to extract the attribute information of the face. The attribute information of the face is fused into the face features extracted by the original P-Net through the attribute information fusion layer, thereby enhancing the face feature representation and improving the detection accuracy of the new P-Net. This reduces the number of generated face candidate frames, reduces the overall computing resources of the face detection model, and improves the detection speed. This improves the existing technology of using the MTCNN network for face detection, which has a low accuracy rate and affects the speed of face detection due to the simple structure of the P-Net. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flowchart of a face detection model training method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a new P-Net provided in an embodiment of the present application;

[0050] Figure 3 A flowchart of a face detection method provided in an embodiment of the present application;

[0051] Figure 4A structural diagram of a face detection model training device provided in an embodiment of the present application;

[0052] Figure 5 A structural diagram of a face detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application provides a face detection model training method, a face detection method and related devices, which are used to improve the existing technology of using the MTCNN network for face detection. Due to the simple structure of P-Net, the accuracy of face detection is low and the speed of face detection is affected.

[0054] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] For easier understanding, see Figure 1 , an embodiment of a face detection model training method provided by the present application includes:

[0056] Step 101: Add a face attribute network branch after the first convolutional layer of P-Net in the MTCNN network, and add an attribute fusion layer between the second and third convolutional layers of P-Net to construct a new P-Net.

[0057] The P-Net in the original MTCNN network is composed of three convolutional layers and a face prediction layer in cascade. In the embodiment of the present application, a face attribute network branch is added after the first convolutional layer of P-Net, and an attribute fusion layer is added between the second and third convolutional layers of P-Net. The face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence. The fourth convolutional layer is connected to the attribute information fusion layer, and the output of the third convolutional layer is connected to the input of the face prediction layer. The structure of the constructed new P-Net can be referred to Figure 2 .

[0058] Step 102: Train the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using the face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net. The label information of the training samples in the face detection dataset includes face information and face attribute information.

[0059] In the embodiment of the present application, the new P-Net, R-Net, and O-Net networks are trained in stages. First, the new P-Net is trained, then the R-Net is trained, and finally the O-Net is trained. The network structures of R-Net and O-Net remain consistent with the R-Net and O-Net in the original MTCNN network. The training process of the new P-Net is as follows:

[0060] The training samples in the face detection dataset are input into the new P-Net. The first convolution layer in the new P-Net performs the first convolution on the training samples to obtain the first face feature map. The second convolution layer in the new P-Net performs the second convolution on the first face feature map to obtain the second face feature map. The fourth convolution layer in the new P-Net performs the third convolution on the first face feature map to obtain the face attribute feature map. The attribute information fusion layer performs feature fusion on the second face feature map and the face attribute feature map to obtain the fusion feature, and the third convolution layer and the face prediction layer perform the third convolution on the first face feature map to obtain the fusion feature. The face prediction layer performs the fourth convolution processing and face prediction on the fusion features in sequence to obtain the face prediction results corresponding to the training samples; the face attribute feature map is sequentially pooled and predicted through the global average pooling layer and the attribute prediction layer to obtain the face attribute prediction results corresponding to the training samples; the face loss value is calculated according to the face prediction results of the training samples and the face information in the label information, and the attribute loss value is calculated according to the face attribute prediction results of the training samples and the face attribute information in the label information, and the network parameters of the new P-Net are updated by the face loss value and the attribute loss value until the new P-Net converges.

[0061] The face detection dataset can be obtained from the existing face dataset, such as the WIDER FACE dataset and the CelebA dataset. Since this application requires the attribute features of the face (eyes, mouth, nose, etc.), during training, it is necessary not only to make face labels for the face detection dataset, but also to make face attribute labels for the face detection dataset. According to the position of the key points of the face in the face dataset, the face attribute information such as eyes, nose, and mouth can be located, and the corresponding positions can be marked with corresponding labels, and then used for model training. Compared with the traditional MTCNN network training method, the embodiment of the present application not only uses the key point position information of the face attributes, but also uses the label information of the corresponding positions. By enriching the supervision signal of the dataset, it can better guide network learning.

[0062] The convolution kernel sizes of the first convolution layer conv1, the second convolution layer conv2, the third convolution layer conv3 and the fourth convolution layer conv4 of the new P-Net in the embodiment of the present application are 3×3×10, 3×3×16, 3×3×32 and 3×3×3 respectively; the input image is feature extracted by conv1 and conv2 to obtain facial feature representation. The first facial feature map obtained after convolution of conv1 is convolved with conv4 to obtain a feature representation of facial attributes. The convolution kernel of conv4 is set to 3×3×3 in order to keep the size of the extracted facial attribute feature map consistent with the size of the second facial feature map extracted by conv2 for subsequent point-by-point operations. At the same time, the number of channels is set to 3 in order to obtain feature representations of the three facial attributes of eyes, nose and mouth.

[0063] The global average pooling layer is then used to perform global average pooling (GAP) on the facial attribute feature map to obtain the feature representation of the facial attributes. The global average pooling layer globally averages the facial attribute feature map of each channel to obtain the feature representation of that channel, mapping the multi-channel facial attribute feature map to a one-dimensional column vector. The introduction of the global average pooling layer greatly reduces the model parameters and reduces the model complexity. The eigenvalues of each dimension in the extracted feature vector are all floating-point values normalized to [0,1], representing the similarity score of the corresponding attribute. During training, the prepared facial attribute labels are used to judge the confidence that the current region contains facial attributes.

[0064] In order to further improve the representation of facial attribute features, the embodiment of the present application also introduces a maximum pooling layer in the new P-Net. The input end of the maximum pooling layer is connected to the output end of the fourth convolutional layer, and the output end is connected to the input end of the attribute information fusion layer. Using the maximum pooling method (Max-pooling), the multi-channel facial attribute feature map is fused to obtain a single-channel facial attribute weight map. Through the maximum pooling method, the significant feature information of the relevant area can be effectively extracted, and the robustness of the feature representation can be improved. Through the feature map extracted along the channel dimension, each pixel value represents the possibility of the presence of facial attributes at the current pixel position.

[0065] The attribute information fusion layer then performs a point-by-point multiplication of the second facial feature map and the facial attribute weight map to generate fused features. This adds information about facial attributes to the extracted facial feature representation. By introducing facial attribute information as a supervisory signal, the network's learning is effectively guided, enhancing the representation of facial features and improving the recognition accuracy of the network model.

[0066] Finally, the loss value is calculated based on the output results of the two networks and the corresponding label information, and the network parameters are reversely updated using the loss value until the network converges to obtain a trained new P-Net.

[0067] After training the new P-Net, remove the global average pooling layer, attribute prediction layer, and face prediction layer from the new P-Net, input the training samples from the face detection dataset into the trained new P-Net, and crop the corresponding training samples according to the candidate boxes output by the face prediction layer to obtain training samples for training the R-Net. After obtaining the trained R-Net, crop the R-Net training samples again according to the candidate boxes output by the R-Net to obtain training samples for training the O-Net. Then, train the O-Net to obtain the trained O-Net. The training process of the R-Net and O-Net can refer to the training process of the R-Net and O-Net in the MTCNN network in the prior art, and will not be repeated here.

[0068] In the embodiments of the present application, taking into account the constraints of actual scenarios, a new P-Net based on multi-attribute fusion is proposed, which improves the face detection effect while reducing the waste of computing resources and storage resources as much as possible; uses facial attribute information as supervision information to guide the learning of the network, thereby improving the face detection accuracy of the network; and by improving the accuracy of the front part of the network in the cascade model, reduces the redundant calculation of the subsequent complex network, thereby achieving the purpose of saving computing resources.

[0069] Step 103: Combine the trained new P-Net, the trained R-Net, and the trained O-Net to construct a face detection model.

[0070] After obtaining the trained new P-Net, the trained R-Net and the trained O-Net, the global average pooling layer, the attribute prediction layer and the face prediction layer in the trained new P-Net, as well as the face prediction layer in the trained R-Net are removed, and then the trained new P-Net, the trained R-Net and the trained O-Net are cascaded in sequence to obtain a multi-attribute fusion face detection model.

[0071] In an embodiment of the present application, the P-Net in the original MTCNN network is improved, and a face attribute network branch is added after the first convolutional layer of the P-Net to extract face attribute information. The face attribute information is fused into the face features extracted by the original P-Net through the attribute information fusion layer, thereby enhancing the face feature representation and improving the detection accuracy of the new P-Net, thereby reducing the number of generated face candidate frames, reducing the overall computing resources of the face detection model, and improving the detection speed. This improves the existing technology of using the MTCNN network for face detection, which has a low accuracy rate and affects the face detection speed due to the simple structure of the P-Net.

[0072] The above is an embodiment of a face detection model training method provided by this application, and the following is an embodiment of a face detection method provided by this application.

[0073] Please refer to Figure 3 , an embodiment of the present application provides a face detection method, comprising:

[0074] Step 201: Obtain a face image to be detected.

[0075] The face image to be detected can be obtained through an image acquisition device.

[0076] Step 202: Input the image to be detected into the face detection model to perform face detection, and obtain the face detection result of the image to be detected.

[0077] The face detection model in the embodiment of the present application is obtained by the face detection model training method in the above method embodiment. The image to be detected is input into the face detection model. The trained new P-Net is used to extract facial features from the face image to be detected, and several face candidate windows are obtained. The trained R-Net is used to filter and optimize the face candidate windows to obtain optimized face candidate windows. The trained O-Net is used to perform face detection on the optimized face candidate windows, and the face detection results are output.

[0078] In the embodiment of the present application, taking into account the constraints of actual scenarios, a face detection model based on multi-attribute fusion is adopted to improve the face detection effect while minimizing the waste of computing resources and storage resources as much as possible; by improving the accuracy of the front part of the network in the cascade model, the redundant calculation of the subsequent complex network is reduced, thereby achieving the purpose of saving computing resources.

[0079] The above is an embodiment of a face detection method provided by the present application, and the following is an embodiment of a face detection model training device provided by the present application.

[0080] Please refer to Figure 4, an embodiment of the present application provides a face detection model training device, comprising:

[0081] The first construction unit is configured to add a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and to add an attribute fusion layer between the second and third convolutional layers of the P-Net, to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output of the third convolutional layer is connected to the input of the face prediction layer;

[0082] A training unit is used to train the new P-Net, the R-Net and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net and a trained O-Net. The label information of the training samples in the face detection dataset includes face information and face attribute information.

[0083] The second construction unit is used to combine the trained new P-Net, the trained R-Net and the trained O-Net to construct a face detection model.

[0084] As a further improvement, the training process of the new P-Net is:

[0085] Input the training samples in the face detection dataset into the new P-Net, perform the first convolution processing on the training samples through the first convolution layer in the new P-Net, and obtain the first face feature map;

[0086] Perform a second convolution on the first facial feature map through the second convolution layer in the new P-Net to obtain a second facial feature map;

[0087] The first face feature map is subjected to the third convolution process by the fourth convolution layer in the new P-Net to obtain the face attribute feature map;

[0088] The second face feature map and the face attribute feature map are fused through the attribute information fusion layer to obtain fused features, and the fused features are sequentially subjected to the fourth convolution processing and face prediction through the third convolution layer and the face prediction layer to obtain the face prediction result corresponding to the training sample;

[0089] The facial attribute feature map is pooled and predicted in sequence through the global average pooling layer and the attribute prediction layer to obtain the facial attribute prediction results corresponding to the training samples;

[0090] The face loss value is calculated according to the face prediction results of the training samples and the face information in the label information. The attribute loss value is calculated according to the face attribute prediction results of the training samples and the face attribute information in the label information. The network parameters of the new P-Net are updated by the face loss value and the attribute loss value until the new P-Net converges.

[0091] In an embodiment of the present application, the P-Net in the original MTCNN network is improved, and a face attribute network branch is added after the first convolutional layer of the P-Net to extract face attribute information. The face attribute information is fused into the face features extracted by the original P-Net through the attribute information fusion layer, thereby enhancing the face feature representation and improving the detection accuracy of the new P-Net, thereby reducing the number of generated face candidate frames, reducing the overall computing resources of the face detection model, and improving the detection speed. This improves the existing technology of using the MTCNN network for face detection, which has a low accuracy rate and affects the face detection speed due to the simple structure of the P-Net.

[0092] The above is an embodiment of a face detection model training device provided by the present application, and the following is an embodiment of a face detection device provided by the present application.

[0093] Please refer to Figure 5 , an embodiment of the present application provides a face detection model training device, comprising:

[0094] An acquisition unit, used for acquiring a face image to be detected;

[0095] The detection unit is used to input the image to be detected into the face detection model for face detection to obtain the face detection result of the image to be detected, wherein the face detection model is obtained by the face detection model training method in the above method embodiment.

[0096] In the embodiment of the present application, taking into account the constraints of actual scenarios, a face detection model based on multi-attribute fusion is adopted to improve the face detection effect while minimizing the waste of computing resources and storage resources as much as possible; by improving the accuracy of the front part of the network in the cascade model, the redundant calculation of the subsequent complex network is reduced, thereby achieving the purpose of saving computing resources.

[0097] An embodiment of the present application further provides an electronic device, comprising: a processor and a memory;

[0098] The memory is used to store program codes and transmit the program codes to the processor;

[0099] The processor is used to execute the face detection model training method in the aforementioned face detection model training method embodiment, or execute the face detection method in the aforementioned face detection method embodiment according to the instructions in the program code.

[0100] An embodiment of the present application also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the face detection model training method in the aforementioned face detection model training method embodiment, or execute the face detection method in the aforementioned face detection method embodiment.

[0101] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0104] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), random access memory (English full name: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program code.

[0106] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A face detection model training method, characterized in that: include: Adding a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and adding an attribute information fusion layer between the second and third convolutional layers of the P-Net to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output end of the third convolutional layer is connected to the input end of the face prediction layer; Training the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net, wherein the label information of the training samples in the face detection dataset includes face information and face attribute information; Combining the trained new P-Net, the trained R-Net, and the trained O-Net to construct a face detection model; The training process of the new P-Net is: Inputting the training samples in the face detection dataset into the new P-Net, performing a first convolution process on the training samples through the first convolution layer in the new P-Net, and obtaining a first facial feature map; Performing a second convolution process on the first facial feature map through the second convolution layer in the new P-Net to obtain a second facial feature map; Performing a third convolution process on the first facial feature map through the fourth convolution layer in the new P-Net to obtain a facial attribute feature map; Performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature, and sequentially performing a fourth convolution process and face prediction on the fused feature through the third convolution layer and the face prediction layer to obtain a face prediction result corresponding to the training sample; Performing pooling processing and facial attribute prediction on the facial attribute feature map in sequence through the global average pooling layer and the attribute prediction layer to obtain facial attribute prediction results corresponding to the training sample; Calculating a face loss value based on the face prediction result of the training sample and the face information in the label information, calculating an attribute loss value based on the face attribute prediction result of the training sample and the face attribute information in the label information, and updating the network parameters of the new P-Net by using the face loss value and the attribute loss value until the new P-Net converges; The new P-Net further includes a maximum pooling layer, wherein the input end of the maximum pooling layer is connected to the output end of the fourth convolutional layer, and the output end is connected to the input end of the attribute information fusion layer; Accordingly, the method further includes: performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature; Performing a maximum pooling operation on the facial attribute feature map through the maximum pooling layer to obtain a facial attribute weight map; each pixel value in the facial attribute weight map represents the probability of the presence of a facial attribute at each pixel position; The step of fusing the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature includes: The attribute information fusion layer performs a point-by-point multiplication operation on the second facial feature map and the facial attribute weight map to obtain a fused feature.

2. A face detection method, characterized in that: include: Get the face image to be detected; The image to be detected is input into a face detection model for face detection to obtain a face detection result of the image to be detected, wherein the face detection model is obtained by the face detection model training method according to claim 1.

3. A face detection model training device, characterized in that: include: A first construction unit is configured to add a face attribute network branch after the first convolutional layer of the P-Net in the MTCNN network, and to add an attribute information fusion layer between the second and third convolutional layers of the P-Net to construct a new P-Net, wherein the face attribute network branch includes a fourth convolutional layer, a global average pooling layer, and an attribute prediction layer connected in sequence, the fourth convolutional layer is connected to the attribute information fusion layer, and the output end of the third convolutional layer is connected to the input end of the face prediction layer; a training unit, configured to train the new P-Net, the R-Net, and the O-Net in the MTCNN network in stages using a face detection dataset to obtain a trained new P-Net, a trained R-Net, and a trained O-Net, wherein the label information of the training samples in the face detection dataset includes face information and face attribute information; A second construction unit is configured to construct a face detection model by combining the trained new P-Net, the trained R-Net, and the trained O-Net; The training process of the new P-Net is: Inputting the training samples in the face detection dataset into the new P-Net, performing a first convolution process on the training samples through the first convolution layer in the new P-Net, and obtaining a first facial feature map; Performing a second convolution process on the first facial feature map through the second convolution layer in the new P-Net to obtain a second facial feature map; Performing a third convolution process on the first facial feature map through the fourth convolution layer in the new P-Net to obtain a facial attribute feature map; Performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature, and sequentially performing a fourth convolution process and face prediction on the fused feature through the third convolution layer and the face prediction layer to obtain a face prediction result corresponding to the training sample; Performing pooling processing and facial attribute prediction on the facial attribute feature map in sequence through the global average pooling layer and the attribute prediction layer to obtain facial attribute prediction results corresponding to the training sample; Calculating a face loss value based on the face prediction result of the training sample and the face information in the label information, calculating an attribute loss value based on the face attribute prediction result of the training sample and the face attribute information in the label information, and updating the network parameters of the new P-Net by using the face loss value and the attribute loss value until the new P-Net converges; The new P-Net further includes a maximum pooling layer, wherein the input end of the maximum pooling layer is connected to the output end of the fourth convolutional layer, and the output end is connected to the input end of the attribute information fusion layer; Accordingly, the method further includes: performing feature fusion on the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature; Performing a maximum pooling operation on the facial attribute feature map through the maximum pooling layer to obtain a facial attribute weight map; each pixel value in the facial attribute weight map represents the probability of the presence of a facial attribute at each pixel position; The step of fusing the second facial feature map and the facial attribute feature map through the attribute information fusion layer to obtain a fused feature includes: The attribute information fusion layer performs a point-by-point multiplication operation on the second facial feature map and the facial attribute weight map to obtain a fused feature.

4. A face detection device, characterized in that: include: An acquisition unit, used for acquiring a face image to be detected; A detection unit is used to input the image to be detected into a face detection model for face detection to obtain a face detection result of the image to be detected, wherein the face detection model is obtained by the face detection model training method according to claim 1.

5. An electronic device, characterized in that: include: processor and memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the face detection model training method according to claim 1, or execute the face detection method according to claim 2 according to the instructions in the program code.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the face detection model training method described in claim 1, or execute the face detection method described in claim 2.

Images