A face detection method and device, electronic equipment and storage medium

Abstract

The present application provides a kind of face detection method, device, electronic equipment and storage medium, to the low-light image to be detected that has been carried out image enhancement processing is carried out multi-scale transformation processing;For each pre-training face detection model, the multiple target low-light images to be detected obtained by multi-scale transformation processing are input into the pre-training face detection model, so that the pre-training face detection model carries out face detection on each target low-light image to be detected, obtains the face detection result of each target low-light image to be detected, and the result of merging the face detection result of each target low-light image to be detected is processed, to obtain the target face detection result of pre-training face detection model;The result of merging the target face detection result of each pre-training face detection model is processed by non-maximum suppression, to obtain the face detection result of low-light image to be detected.The present application can improve face detection precision under low-light condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face detection, and more particularly to a face detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, face recognition has gradually become the hottest topic in the field of vision research. Face recognition has important application value in the fields of security monitoring, person-certificate comparison, face unlocking, human-computer interaction, and face special effects. The basis of face recognition is face detection.

[0003] With the development of deep learning, face detection has made great progress and can achieve very excellent performance in most scenarios. However, outdoor scenes are easily affected by complex visual degradation in adverse weather, for example, face detection tasks in low-light conditions can cause reduced visibility and loss of signal details, resulting in missed detection and false detection. SUMMARY

[0004] Therefore, the present application provides a face detection method, device, electronic device and storage medium to improve face detection accuracy in low-light conditions.

[0005] The first aspect of the present application discloses a face detection method, comprising:

[0006] obtaining a low-light image to be detected, and performing image enhancement processing on the low-light image to be detected;

[0007] performing multi-scale transformation processing on the low-light image to be detected that has undergone image enhancement processing to obtain a plurality of target low-light images to be detected, wherein the size of each target low-light image to be detected is different;

[0008] For each pre-trained face detection model, inputting the plurality of target low-light images to be detected into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each target low-light image to be detected to obtain a face detection result of each target low-light image to be detected, and processing the result of merging the face detection results of each target low-light image to be detected to obtain a target face detection result of the pre-trained face detection model;

[0009] performing non-maximum suppression processing on the result of merging the target face detection results of each pre-trained face detection model to obtain a face detection result of the low-light image to be detected.

[0010] Optionally, the obtaining a low-light image to be detected, and performing image enhancement processing on the low-light image to be detected comprises:

[0011] obtaining a low-light image to be detected;

[0012] performing image enhancement processing on the low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR).

[0013] Optionally, the backbone networks of each of the pre-trained face detection models are different, and each of the pre-trained face detection models is trained by using a historical low-light image to be detected that has undergone image enhancement processing.

[0014] Optionally, the process of training the face detection model to be trained by using the historical low-light image to be detected that has undergone image enhancement processing to obtain the pre-trained face detection model comprises:

[0015] obtaining a plurality of training data, each of which is obtained by performing image enhancement processing on a historical low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR) and a small face data enhancement algorithm;

[0016] For each of the training data, the training data is input into the face detection model to be trained, so that the face detection model to be trained performs face detection on the training data to obtain a face detection result of the historical low-light image to be detected, a corresponding loss function is calculated according to the face detection result of the historical low-light image to be detected, so as to adjust parameters of the face detection model to be trained according to the loss function, until the face detection model to be trained converges, and a face detection model is obtained.

[0017] Optionally, if the backbone network of the face detection model to be trained is a ResNet50 neural network, the ResNet50 neural network comprises an input layer, three stages, a convolutional layer connected to each of the stages, and a context module connected to each of the convolutional layers, and the process of training the face detection model to be trained by using the historical low-light image to be detected that has undergone image enhancement processing to obtain the pre-trained face detection model comprises:

[0018] inputting the training data into the ResNet50 neural network to be trained through the input layer;

[0019] processing the training data by using the three stages respectively to obtain a feature map corresponding to each of the stages;

[0020] The feature map corresponding to each stage is reduced by using the corresponding convolution layer of each stage, to obtain a target feature map corresponding to each convolution layer, and a fusion result obtained by performing multi-scale fusion processing on each target feature map is input into the context module corresponding to each convolution layer;

[0021] Based on the feature map obtained by processing the input fusion result by each context module, a multi-task loss function of each context module is calculated, and the parameters of the ResNet50 neural network to be trained are adjusted by using the multi-task loss function of each context module, until the ResNet50 neural network to be trained converges, to obtain a face detection model.

[0022] Optionally, the face detection result at least includes at least one face frame and a confidence corresponding to each face frame, and the method further comprises:

[0023] It is judged whether the confidence corresponding to each face frame is greater than a preset confidence threshold;

[0024] If the confidence corresponding to each face frame is greater than the preset confidence threshold, the face detection result is determined as the final face detection result of the low-light image to be detected.

[0025] The second aspect of the present application discloses a face detection device, which comprises:

[0026] A first acquisition unit is configured to acquire a low-light image to be detected and perform image enhancement processing on the low-light image to be detected.

[0027] A multi-scale transformation processing unit is configured to perform multi-scale transformation processing on the low-light image to be detected that has been subjected to image enhancement processing, to obtain a plurality of target low-light images to be detected, wherein the size of each target low-light image to be detected is different.

[0028] A face detection unit is configured to input the plurality of target low-light images to be detected into each pre-trained face detection model, so that each pre-trained face detection model performs face detection on each target low-light image to be detected, to obtain a face detection result of each target low-light image to be detected, and process the result of merging the face detection result of each target low-light image to be detected, to obtain a target face detection result of each pre-trained face detection model.

[0029] A non-maximum suppression processing unit is configured to perform non-maximum suppression processing on the result of merging the target face detection results of the respective pre-trained face detection models, so as to obtain the face detection result of the to-be-detected low-light image.

[0030] Optionally, the first acquisition unit comprises:

[0031] A second acquisition unit is configured to acquire a to-be-detected low-light image.

[0032] An image enhancement processing unit is configured to perform image enhancement processing on the to-be-detected low-light image by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR).

[0033] The third aspect of the present application discloses an electronic device, comprising: a processor and a memory, the processor and the memory are connected through a communication bus; wherein the processor is used for calling and executing the program stored in the memory; the memory is used for storing the program, and the program is used for realizing the face detection method disclosed in the first aspect of the present application.

[0034] The fourth aspect of the present application discloses a computer readable storage medium, and the computer readable storage medium stores computer executable instructions, and the computer executable instructions are used for executing the face detection method disclosed in the first aspect of the present application.

[0035] The present application provides a face detection method and device, electronic equipment and storage medium. The face detection method comprises the following steps: obtaining a low-light image to be detected, and performing image enhancement processing on the low-light image to be detected; performing multi-scale transformation processing on the low-light image to be detected which has been subjected to image enhancement processing, to obtain a plurality of target low-light images to be detected, wherein the size of each target low-light image to be detected is different; for each pre-trained face detection model, inputting the plurality of target low-light images to be detected into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each target low-light image to be detected, to obtain a face detection result of each target low-light image to be detected, and processing a result obtained by merging the face detection results of each target low-light image to be detected, to obtain a target face detection result of the pre-trained face detection model; and performing non-maximum suppression processing on the target face detection results of each pre-trained face detection model, to obtain a face detection result of the low-light image to be detected. The technical scheme provided by the present application trains a face detection model by using historical low-light images to be detected which have been subjected to image enhancement processing, so that the model can better learn the characteristics of faces in a low-light outdoor scene, greatly improves the performance of the model in this scene, and then after obtaining a plurality of target low-light images to be detected with different sizes by performing image enhancement processing and multi-scale transformation processing on the low-light image to be detected, inputting the plurality of target low-light images to be detected with different sizes into each face detection model trained by using historical low-light images to be detected which have been subjected to image enhancement processing, and performing face detection by using each face detection model, the accuracy of face detection in a low-light condition can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0037] Figure 1 A flowchart of a face detection method provided by an embodiment of the present application;

[0038] Figure 2 A structural diagram of a ResNet50 neural network provided by an embodiment of the present application;

[0039] Figure 3 An example diagram of a face detection method provided by an embodiment of the present application;

[0040] Figure 4 A structural diagram of a face detection device provided by an embodiment of the present application;

[0041] Figure 5 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0043] The term "comprising" and variations thereof as used in the present document are open-ended, that is, "comprising but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions of other terms will be given in the following description.

[0044] It should be noted that the "first", "second", and the like concepts mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0045] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0046] Reference is made to Figure 1 A flowchart of a face detection method provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0047] S101: Obtain a low-light image to be detected, and perform image enhancement processing on the low-light image to be detected.

[0048] In the process of specifically performing step S101, the low-light image to be detected is obtained, and a multi-scale retinal enhancement algorithm with color restoration (Multi-Scale Retinex with Color Restoration, MSRCR) is used to perform image enhancement processing on the obtained low-light image to be detected.

[0049] The MSRCR is a widely used image enhancement method based on the Retinex theory. The Retinex is established on the basis of scientific experiments and scientific analysis, and can adaptively enhance various types of images. In the practical process of the application, a parameter suitable for low-light conditions can be set to achieve a good image enhancement effect.

[0050] S102: Perform multi-scale transformation processing on the image-enhanced low-light image to be detected to obtain a plurality of target low-light images to be detected, wherein each target low-light image to be detected has a different size.

[0051] In the process of specifically performing step S102, after the low-light image to be detected is image-enhanced, the image-enhanced low-light image to be detected is processed by a multi-level multi-scale fusion strategy to obtain a plurality of target low-light images to be detected with different sizes.

[0052] S103: For each pre-trained face detection model, input the plurality of target low-light images to be detected into the pre-trained face detection model to enable the pre-trained face detection model to detect faces in each target low-light image to be detected, obtain a face detection result of each target low-light image to be detected, and process the result of merging the face detection results of each target low-light image to be detected to obtain a target face detection result of the pre-trained face detection model.

[0053] In the process of step S103, a plurality of face detection models are pre-trained, each of which is trained using a historical low-light image to be detected that has been image-enhanced, wherein the backbone network of each pre-trained face detection model is different.

[0054] Specifically, the process of training a face detection model using a historical low-light image to be detected that has been image-enhanced includes: obtaining a plurality of historical low-light images to be detected, and using a multi-scale retinal enhancement algorithm MSRCR with color restoration and a small face data augmentation algorithm to image-enhance each historical low-light image to be detected to obtain training data corresponding to each historical low-light image to be detected.

[0055] For each training data, input the training data into a to-be-trained face detection model to enable the to-be-trained face detection model to detect faces in the training data, obtain a face detection result of the historical low-light image to be detected, calculate a corresponding loss function according to the face detection result of the historical low-light image to be detected, adjust parameters of the to-be-trained face detection model according to the loss function, and obtain a face detection model until the to-be-trained face detection model converges.

[0056] In the embodiment of the present application, the specific process of processing the historical low-light image to be detected by using the small face data enhancement algorithm can be: a face in the historical low-light image to be detected is randomly selected, the size of the selected face is denoted as s, and a value a1 with the minimum absolute value difference from s is selected from {16, 32, 64, 128, 256, 512}, and the index of a1 is denoted as ind1, then an index ind2 is randomly selected from [1, ind1], the value corresponding to the index ind2 is denoted as a2, scale = a2 / s is calculated, the image is enlarged or reduced according to scale, and then the image after scaling is randomly cropped and filled to obtain an image with a size of 640*640. At this time, the image has undergone the first step of data enhancement, and we need to perform a cutting operation again. The specific method is to randomly select a value from {0.3, 0.45, 0.6, 0.8, 1.0}, for example, 0.6 is selected, then 0.6*640 is calculated, a 384*384 size image block containing the face is cropped from the 640*640 image, and the image block is enlarged to an image with a size of 640*640.

[0057] In the embodiment of the present application, the backbone network of the face detection model can be a ResNet50 neural network, the ResNet50 neural network includes an input layer, 3 stages, a convolutional layer connected with each stage, and a context module connected with each convolutional layer, as shown in Figure 2 .

[0058] If the backbone network of the face detection model to be trained is a ResNet50 neural network, the process of training the face detection model to be trained by using the historical low-light image to be detected subjected to image enhancement processing to obtain the pre-trained face detection model can be: training data with a size of 640*640*3 are obtained, and the training data are input to the ResNet50 neural network to be trained through the input layer of the ResNet50 neural network to be trained, the input training data are processed by using 3 different stages respectively, and a first stage feature map P1 with a size of 80*80*256, a second stage feature map P2 with a size of 40*40*1024, and a third stage feature map P3 with a size of 20*20*2048 are obtained.

[0059] The channel number of the feature map P1 is reduced to 256 by using the convolution layer corresponding to the first stage to obtain a target feature map P1' with a size of 80*80*256, the channel number of the feature map P2 is reduced to 256 by using the convolution layer corresponding to the second stage to obtain a target feature map P2' with a size of 40*40*256, and the channel number of the feature map P3 is reduced to 256 by using the convolution layer corresponding to the third stage to obtain a target feature map P3' with a size of 20*20*256; the target feature map P3' is fused with an empty target feature map to obtain a fusion result C3, that is, the target feature map P3' is directly taken as the fusion result C3 and input into the context module corresponding to the convolution layer corresponding to the third stage, so that the context module processes the fusion result C3 and outputs a feature map maintaining the original size; the target feature map P2' is fused with the fusion result C3 to obtain a fusion result C2, and the fusion result C2 is input into the context module corresponding to the convolution layer corresponding to the second stage, so that the context module processes the fusion result C2 and outputs a feature map maintaining the original size; the target feature map P1' is fused with the fusion result C2 to obtain a fusion result C1, and the fusion result C1 is input into the context module corresponding to the convolution layer corresponding to the first stage, so that the context module processes the fusion result C1 and outputs a feature map maintaining the original size. The context module is composed of a plurality of convolution layers with larger kernel sizes such as 5*5 and 7*7, which can introduce more context information by expanding the receptive field on the basis of maintaining the size of the output feature map.

[0060] The regression box position loss and the regression box classification loss of each context module are calculated according to the feature map maintaining the original size output by the context module, and then the multi-task loss function of the context module is calculated according to the regression box position loss and the regression box classification loss of the context module. Taking C3 as an example, the size of the feature map of C3 is 20*20*256, which remains unchanged after passing through the corresponding context module. In order to calculate the regression box position loss, the feature map size needs to be changed to 20*20*4 by using a detection head (a 1*1 convolution operation), where the channel dimension is 4, representing the position information of the regression box; similarly, in order to calculate the regression box classification loss, the feature map size needs to be changed to 20*20*2 by using a classification head (a 1*1 convolution operation), where the channel dimension is 2, representing the classification information (whether it is a face or not) of the regression box. Finally, the regression box position loss function and the regression box classification loss function are added to obtain the multi-task loss function of the context module, as shown in formula (1).

[0061] After the multi-task loss function of each context module is calculated, the parameters of the ResNet50 neural network to be trained are adjusted using the multi-task loss function of each context module until the ResNet50 neural network to be trained converges, and a face detection model is obtained.

[0062]

[0063] wherein i represents an index of a regression box, P i represents a probability of whether the regression box is a face, P i * is a label of the regression box, wherein 1 represents a face and 0 represents a non-face. i is position information of the regression box, t i * is position information of the face regression box. cls The softmax loss function of two categories (face and background) is used, and the smoothL1 loss function is used for Lreg, which is multiplied by P i * to ensure that the position loss function is only calculated for the face regression box, and λ is used to weigh the regression box position loss function and the regression box classification loss function.

[0064] Further, in the embodiments of the present application, after the face detection model is trained, the trained face detection model can be further tested. Specifically, a plurality of historical low-light images to be detected are obtained, and each historical low-light image to be detected is subjected to image enhancement processing using a multi-scale retinal enhancement algorithm MSRCR with color restoration. The historical low-light image to be detected subjected to image enhancement processing is subjected to multi-scale transformation processing, and test data of each historical low-light image to be detected is obtained, wherein the test data of each historical low-light image to be detected includes a plurality of historical low-light images to be detected subjected to image enhancement processing with different sizes.

[0065] For each test data, the pre-trained face detection model of the test data is pre-trained, so that the pre-trained face detection model processes each historical low-light image to be detected of different sizes subjected to image enhancement processing, obtains the face detection result of each historical low-light image to be detected of different sizes subjected to image enhancement processing, and performs non-maximum suppression processing on the result of merging each face detection result to obtain the face detection result of the historical low-light image to be detected corresponding to the test data. The face detection result of the historical low-light image to be detected corresponding to the test data, the actual face detection calculation of the historical low-light image to be detected corresponding to the test data, the average value (mean Average Precision, mAP) of each category AP is calculated, and when the mAP of the face detection model reaches the preset mAP, it is determined that the performance of the face detection model meets the standard.

[0066] wherein AP is the most commonly used evaluation index in target detection, and mean Average Precision (mAP) is the average value of each category AP. Since there is only one category in the face detection task, i.e. face. AP is the area under the Precision-Recall curve, and the larger the area, the higher the AP, i.e. the better the model performance, wherein the range of AP is 0-1. The preset mAP can be 0.6944, which can be set according to actual application, and the embodiments of the present application are not limited.

[0067] In the process of specifically executing step S103, a plurality of target low-light images to be detected are input into each pre-trained face detection model. For each pre-trained face detection model, the pre-trained face detection model performs face detection on each target low-light image to be detected to obtain the face detection result of each target low-light image to be detected. The face detection results of each target low-light image to be detected are merged, and the merged results are subjected to non-maximum suppression processing to obtain the target face detection result of the pre-trained face detection model.

[0068] S104: The results of merging the target face detection results of each pre-trained face detection model are subjected to non-maximum suppression processing to obtain the face detection result of the low-light image to be detected.

[0069] In the process of specifically executing step S104, after obtaining the target face detection result of each pre-trained face detection model, the target face detection results of each pre-trained face detection model are merged, and the merged results are subjected to non-maximum suppression processing to obtain the face detection result of the low-light image to be detected.

[0070] Furthermore, in this embodiment, the face detection result of the low-light image to be detected includes at least one face bounding box, the position information of each face bounding box, and the confidence score of each face bounding box. It can then be arbitrarily determined whether the confidence score corresponding to each face bounding box is greater than a preset confidence threshold. If the confidence score corresponding to each face bounding box is greater than the preset confidence threshold, the face detection result is determined as the final face detection result of the low-light image to be detected. If the confidence score corresponding to any face bounding box is not greater than the preset confidence threshold, then face detection is performed again on the low-light image to be detected.

[0071] This invention provides a face detection method, which involves acquiring a low-light image to be detected and performing image enhancement processing on the low-light image; performing multi-scale transformation processing on the image-enhanced low-light image to obtain multiple target low-light images, each with a different size; for each pre-trained face detection model, inputting the multiple target low-light images to the pre-trained face detection model to perform face detection on each target low-light image, obtaining the face detection result for each target low-light image; processing the merged face detection results of each target low-light image to obtain the target face detection result of the pre-trained face detection model; and performing non-maximum suppression processing on the target face detection results of each pre-trained face detection model to obtain the face detection result of the low-light image to be detected. The technical solution provided by this invention utilizes historical low-light images of the target subject to be detected, after image enhancement processing, to train a face detection model. This allows the model to better learn the characteristics of faces in low-light outdoor scenes, greatly improving the model's performance in this scenario. Furthermore, after performing image enhancement processing and multi-scale transformation processing on the low-light images of the target subject to be detected to obtain multiple target low-light images of different sizes, these multiple target low-light images of different sizes are input, and each face detection model trained using the historical low-light images of the target subject to be detected after image enhancement processing is used to perform face detection, thereby improving the accuracy of face detection under low-light conditions.

[0072] To better understand the above content, examples are provided below.

[0073] For example, such as Figure 3 As shown, three face detection models were pre-trained to acquire low-light images to be detected (such as...). Figure 3 The input image is processed by performing image enhancement on the low-light image to be detected. The resolution of the enhanced image is then increased by 2 times and 3 times respectively to obtain two larger-scale low-light images of the target to be detected, namely, the target low-light image of size 1 and the target low-light image of size 2.

[0074] input the target low-light image with the size of size 1 and the target low-light image with the size of size 2 into the pre-trained face detection model one, so that the pre-trained face detection model one performs face detection on each target low-light image to obtain a face detection result of each target low-light image, and performs non-maximum suppression on a result of combining the face detection results of each target low-light image to obtain a target face detection result of the pre-trained face detection model one.

[0075] input the target low-light image with the size of size 1 and the target low-light image with the size of size 2 into the pre-trained face detection model two, so that the pre-trained face detection model two performs face detection on each target low-light image to obtain a face detection result of each target low-light image, and performs non-maximum suppression on a result of combining the face detection results of each target low-light image to obtain a target face detection result of the pre-trained face detection model two.

[0076] input the target low-light image with the size of size 1 and the target low-light image with the size of size 2 into the pre-trained face detection model three, so that the pre-trained face detection model three performs face detection on each target low-light image to obtain a face detection result of each target low-light image, and performs non-maximum suppression on a result of combining the face detection results of each target low-light image to obtain a target face detection result of the pre-trained face detection model three.

[0077] After obtaining the target face detection result of each pre-trained face detection model, the target face detection results of the pre-trained face detection models are combined, and non-maximum suppression is performed on the combined result to obtain a face detection result of the target low-light image.

[0078] Based on the face detection method disclosed in the embodiments of the present application, the embodiments of the present application also correspondingly disclose a face detection device, as shown in the accompanying drawings, the face detection device comprises: Figure 4

[0079] The first acquisition unit 41 is configured to acquire the target low-light image and perform image enhancement processing on the target low-light image.

[0080] The multi-scale transformation processing unit 42 is configured to perform multi-scale transformation processing on the target low-light image subjected to the image enhancement processing to obtain a plurality of target low-light images, wherein the size of each target low-light image is different.

[0081] ​The face detection unit 43 is configured to, for each pre-trained face detection model, input the plurality of target low-light images to be detected into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each target low-light image to be detected to obtain a face detection result of each target low-light image to be detected, and processes a result of merging the face detection results of each target low-light image to be detected to obtain a target face detection result of the pre-trained face detection model;

[0082] The non-maximum suppression processing unit 44 is configured to perform non-maximum suppression processing on the result of merging the target face detection results of each pre-trained face detection model to obtain a face detection result of the low-light image to be detected.

[0083] The specific principles and execution processes of each unit in the face detection device disclosed in the embodiments of the present application are the same as those in the face detection method disclosed in the embodiments of the present application, and can be referred to the corresponding parts in the face detection method disclosed in the embodiments of the present application, which will not be repeated here.

[0084] The present application provides a face detection device, acquires a low-light image to be detected, and performs image enhancement processing on the low-light image to be detected; performs multi-scale transformation processing on the low-light image to be detected that has been subjected to image enhancement processing to obtain a plurality of target low-light images to be detected, wherein the size of each target low-light image to be detected is different; for each pre-trained face detection model, input the plurality of target low-light images to be detected into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each target low-light image to be detected to obtain a face detection result of each target low-light image to be detected, and processes a result of merging the face detection results of each target low-light image to be detected to obtain a target face detection result of the pre-trained face detection model; and perform non-maximum suppression processing on the target face detection results of each pre-trained face detection model to obtain a face detection result of the low-light image to be detected. The technical scheme provided by the present application trains a face detection model using historical low-light images to be detected that have been subjected to image enhancement processing, which enables the model to better learn the characteristics of faces in a low-light outdoor scene, greatly improves the performance of the model in this scene, and then after obtaining a plurality of target low-light images to be detected of different sizes through image enhancement processing and multi-scale transformation processing on the low-light image to be detected, inputting the plurality of target low-light images to be detected of different sizes, and performing face detection using each face detection model trained using historical low-light images to be detected that have been subjected to image enhancement processing, the accuracy of face detection in a low-light condition can be improved.

[0085] Optionally, the first acquisition unit comprises:

[0086] A second acquisition unit is configured to acquire the low-light image to be detected.

[0087] An image enhancement processing unit is configured to perform image enhancement processing on the low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR).

[0088] Optionally, the backbone networks of the pre-trained face detection models are different, and each pre-trained face detection model is obtained by training using the training unit.

[0089] Optionally, the training unit comprises:

[0090] A third acquisition unit is configured to acquire a plurality of training data, each training data being obtained by performing image enhancement processing on a historical low-light image to be detected by using the multi-scale retinal enhancement algorithm with color restoration (MSRCR) and a small face data enhancement algorithm.

[0091] A training subunit is configured to, for each training data, input the training data into the face detection model to be trained, so that the face detection model to be trained performs face detection on the training data to obtain a face detection result of the historical low-light image to be detected, calculate a corresponding loss function according to the face detection result of the historical low-light image to be detected, adjust parameters of the face detection model to be trained according to the loss function, until the face detection model to be trained converges, and obtain the face detection model.

[0092] Optionally, if the backbone network of the face detection model to be trained is a ResNet50 neural network, the ResNet50 neural network comprises an input layer, three stages, a convolutional layer connected to each stage, and a context module connected to each convolutional layer, and the face detection unit comprises:

[0093] An input unit is configured to input the training data into the ResNet50 neural network to be trained through the input layer;

[0094] A first processing unit is configured to process the training data by using the three stages respectively to obtain a feature map corresponding to each stage;

[0095] A second processing unit is configured to perform downscaling processing on the feature map corresponding to each stage by using the convolutional layer corresponding to each stage respectively to obtain a target feature map corresponding to each convolutional layer, and input a fusion result obtained by performing multi-scale fusion processing on each target feature map into the context module corresponding to each convolutional layer;

[0096] The parameter adjustment unit is configured to calculate a multi-task loss function of each context module based on a feature map obtained by processing the fusion result of the input, and adjust parameters of a ResNet50 neural network to be trained of each context module by using the multi-task loss function, until the ResNet50 neural network to be trained converges, and obtain the face detection model.

[0097] Further, the face detection result includes at least one face frame and a confidence corresponding to each face frame, and the face detection device provided by the application further includes:

[0098] The determination unit is configured to determine whether the confidence corresponding to each face frame is greater than a preset confidence threshold.

[0099] The determination unit is configured to determine whether the confidence corresponding to each face frame is greater than a preset confidence threshold.

[0100] The embodiment of the application provides an electronic device, as shown in the figure, Figure 5 The electronic device includes a processor 501 and a memory 502, the memory 502 is used for program codes and data for face detection, and the processor 501 is used for calling program instructions in the memory to execute steps shown in the face detection method in the above embodiment.

[0101] The embodiment of the application provides a storage medium, the storage medium includes a storage program, wherein, when the program runs, the device where the storage medium is located executes the face detection method shown in the above embodiment.

[0102] Each embodiment in the specification is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other, and each embodiment mainly describes the difference from other embodiments. Especially, for the system or system embodiment, since it is basically similar to the method embodiment, it is described more simply, and the related part can be referred to the part of the method embodiment. The system and system embodiment described above are only schematic, and the units described as separate units can be or can not be physically separated, and the units displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to the actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0103] Those skilled in the art will further appreciate that the units and algorithms described in connection with the examples disclosed herein can be embodied directly in hardware, in software, or in a combination of the two. For the sake of brevity, descriptions of a functional nature will generally be used throughout this disclosure to describe the various examples. Those skilled in the art will appreciate that the functionality described can be implemented in either hardware or software, or a combination of the two. Depending upon the particular application, the implementation of the described functionality can vary. Those skilled in the art will recognize the interchangeability of hardware and software under these circumstances, and will appreciate the various ways in which software might be implemented. For the sake of brevity, a mathematical formula will occasionally be used within the specification, with the understanding that the function can be implemented either in hardware or software, depending upon the particular application or design constraints.

[0104] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0105] The preferred embodiments of the application are thus described. It is to be understood that the above description is intended to be illustrative, and not restrictive, of the application. Many other modifications and variations of the preferred embodiments of the application will be apparent to those of ordinary skill in the art in light of the above teachings. It is, therefore, to be understood that changes can be made to the form and details of the application without departing from the spirit and the scope of the application. Accordingly, the disclosure of the present application is intended to be illustrative, but not limiting, of the scope of the application, which is set forth in the following claims.

Claims

1. A face detection method, characterized by, The method comprises: obtaining a low-light image to be detected, and performing image enhancement processing on the low-light image to be detected; performing multi-scale transformation processing on the low-light image to be detected subjected to image enhancement processing to obtain a plurality of target low-light images to be detected, wherein the size of each target low-light image to be detected is different; for each pre-trained face detection model, inputting the plurality of target low-light images to be detected into the pre-trained face detection model to enable the pre-trained face detection model to perform face detection on each target low-light image to be detected, obtain a face detection result of each target low-light image to be detected, and perform processing on a result of merging the face detection results of each target low-light image to be detected to obtain a target face detection result of the pre-trained face detection model; the backbone network of each pre-trained face detection model is different, and each pre-trained face detection model is trained by using historical low-light images to be detected subjected to image enhancement processing; performing non-maximum suppression processing on a result of merging the target face detection results of each pre-trained face detection model to obtain a face detection result of the low-light image to be detected; if the backbone network of a to-be-trained face detection model is a ResNet50 neural network, the ResNet50 neural network comprises an input layer, three stages, a convolutional layer connected with each stage, and a context module connected with each convolutional layer, and the process of training the to-be-trained face detection model by using the historical low-light images to be detected subjected to image enhancement processing to obtain the pre-trained face detection model comprises: inputting training data into the to-be-trained ResNet50 neural network through the input layer; processing the training data by using the three stages respectively to obtain a feature map corresponding to each stage; performing reduction processing on the feature map corresponding to each stage by using the convolutional layer corresponding to each stage respectively to obtain a target feature map corresponding to each convolutional layer, and inputting a fusion result obtained by performing multi-scale fusion processing on each target feature map into the context module corresponding to each convolutional layer; based on a feature map obtained by processing the input fusion result by each context module, calculating a multi-task loss function of each context module, and adjusting parameters of the to-be-trained ResNet50 neural network by using the multi-task loss function of each context module until the to-be-trained ResNet50 neural network reaches convergence to obtain a face detection model.

2. The method of claim 1, wherein, The process of obtaining the low-light image to be detected and performing image enhancement processing on the low-light image to be detected comprises: performing image enhancement processing on the low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR).

3. The method of claim 1, wherein, The process of training the to-be-trained face detection model by using the historical low-light images to be detected subjected to image enhancement processing to obtain the pre-trained face detection model comprises: obtaining a plurality of training data, each of the training data being obtained by performing image enhancement processing on a historical low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR) and a small face data enhancement algorithm; for each of the training data, inputting the training data into a to-be-trained face detection model, so that the to-be-trained face detection model performs face detection on the training data, to obtain a face detection result of the historical low-light image to be detected, calculating a corresponding loss function according to the face detection result of the historical low-light image to be detected, so as to adjust parameters of the to-be-trained face detection model according to the loss function, until the to-be-trained face detection model converges, and obtaining a face detection model.

4. The method of claim 1, wherein, The face detection result at least includes at least one face frame and a confidence corresponding to each face frame, and the method further comprises: determining whether the confidence corresponding to each face frame is greater than a preset confidence threshold; if the confidence corresponding to each face frame is greater than the preset confidence threshold, determining the face detection result as a final face detection result of the low-light image to be detected.

5. A face detection apparatus characterized by comprising: The device comprises: a first obtaining unit configured to obtain a low-light image to be detected and perform image enhancement processing on the low-light image to be detected; a multi-scale transformation processing unit configured to perform multi-scale transformation processing on the low-light image to be detected that has been subjected to the image enhancement processing, to obtain a plurality of target low-light images to be detected, wherein the size of each of the target low-light images to be detected is different; a face detection unit configured to, for each pre-trained face detection model, input the plurality of target low-light images to be detected into the pre-trained face detection model, so that the pre-trained face detection model performs face detection on each of the target low-light images to be detected, to obtain a face detection result of each of the target low-light images to be detected, and process a result of merging the face detection result of each of the target low-light images to be detected, to obtain a target face detection result of the pre-trained face detection model; the backbone network of each of the pre-trained face detection models is different, and each of the pre-trained face detection models is trained by using a historical low-light image to be detected that has been subjected to the image enhancement processing; a non-maximum suppression processing unit configured to perform non-maximum suppression processing on a result of merging the target face detection result of each of the pre-trained face detection models, to obtain a face detection result of the low-light image to be detected; if the backbone network of the to-be-trained face detection model is a ResNet50 neural network, the ResNet50 neural network comprises an input layer, three stages, a convolutional layer connected to each stage, and a context module connected to each convolutional layer, and the face detection unit comprises: an input unit configured to input training data into a to-be-trained ResNet50 neural network through the input layer; a first processing unit configured to process the training data by using the three stages respectively, to obtain a feature map corresponding to each stage; The second processing unit is configured to perform downscaling processing on the feature map corresponding to each stage by using the convolution layer corresponding to each stage respectively, to obtain a target feature map corresponding to each convolution layer, and to input a fusion result obtained by performing multi-scale fusion processing on each target feature map into the context module corresponding to each convolution layer; The parameter adjustment unit is configured to calculate a multi-task loss function of each context module based on a feature map obtained by processing the input fusion result by each context module, and to adjust parameters of the ResNet50 neural network to be trained by using the multi-task loss function of each context module until the ResNet50 neural network to be trained converges, so as to obtain the face detection model.

6. The apparatus of claim 5, wherein, The first obtaining unit comprises: The second obtaining unit is configured to obtain the low-light image to be detected. The image enhancement processing unit is configured to perform image enhancement processing on the low-light image to be detected by using a multi-scale retinal enhancement algorithm with color restoration (MSRCR).

7. An electronic device, comprising: The electronic device comprises a processor and a memory, the memory is configured to store program codes and data of face detection, and the processor is configured to call program instructions in the memory to execute the face detection method in any one of claims 1-4.

8. A storage medium, characterized by The storage medium comprises a program, wherein the program controls a device where the storage medium is located to execute the face detection method in any one of claims 1-4 when the program is running.

Images