Low-light face detection method, device, computer equipment and storage medium

Abstract

Embodiments of the present invention disclose a low-light face detection method, apparatus, computer equipment, and storage medium. The method comprises: acquiring an image to be detected; inputting the image to be detected into a face detection model to perform face detection to obtain a detection result; and outputting the detection result. The face detection model is obtained by training a model composed of a gradient-guided generative adversarial network and a target detection network using a plurality of image pairs consisting of low-light face images and normal-light face images as sample sets. By implementing the method of the embodiment of the present invention, face detection in ultra-low light conditions can be achieved without threshold adjustment, and face detection can be performed for different degrees of low light conditions with high accuracy.

Description

Technical Field

[0001] The present invention relates to a face detection method, and more particularly to a low-light illumination face detection method, device, computer equipment and storage medium. Background Art

[0002] With the development of the intelligent society, facial recognition technology has been applied in many scenarios, such as residential access control systems, train station entrance gates, and exhibition security systems. Facial recognition technology not only provides convenience to people but also ensures their safety. For example, in public places like train stations, using facial recognition to pass through gates can greatly improve traffic efficiency compared to traditional manual methods.

[0003] Face detection is a crucial step in face recognition technology. Face detection is typically performed by capturing images, which are affected by illumination. Direct face detection in low-light images can significantly reduce accuracy. Five approaches are currently available. The first employs an adaptive algorithm based on the OTSU segmentation algorithm for low-light conditions. This algorithm adapts to image acquisition in low-light environments and is validated using an Adaboost classification detector on low-light face images before and after processing. In this algorithm, the OTSU algorithm needs to set a threshold for semantic segmentation, and it has no good compatibility with different degrees of low light, and the effect is poor. Secondly, adboost is a weak classifier, and the recognition effect often deteriorates due to data imbalance; the second is to preprocess the face video image and finally perform face recognition through feature fusion modeling based on the Adaboost face detection algorithm. This algorithm preprocesses the face image based on the traditional machine learning model and adjusts the parameters manually. At the same time, the setting of the parameter threshold of the face detection algorithm based on adboost is not accurate enough for the detection effect; the third is to design the memory classifier based on the impact of illumination changes on face recognition from preprocessing, feature extraction, and use the coarse-grained zoning algorithm to divide the continuous contrast value range into intervals combined with illumination change information, and extract LNMCP face features. The human recognition rate is identified by mapping the fuzzy membership function and calculating the weights. However, this algorithm is based on the nonlinear division layer extracted by dynamic thresholds to extract face features. , there are steps of manual parameter setting, which makes the algorithm model not robust enough and cannot adapt well to abnormal light and reflections in different parts of the face; the fourth is to set the mask of the interference area, determine the number of pixels in the interference area, and compare its grayscale value with the threshold, find the pixels exceeding the threshold, and then judge the number of pixels in the image with grayscale exceeding the threshold and compare it with the number of pixels in the interference area. If it is greater than the number of pixels in the interference area, the pixels with grayscale exceeding the threshold are masked with the set mask template to change the grayscale value of the pixels. However, this method relies too much on the manual selection of threshold parameters for de-reflection processing, and has no good compatibility for reflections of different directions and degrees, and only processes glasses reflections, without considering other lighting effects such as skin diffuse reflection, which makes the algorithm not robust enough; the fifth is to pre-process the grayscale values ​​of pixels, adjust the threshold of the image, and perform weightless fusion by calculating the neighborhood matrix of the image pixels. The grayscale values ​​of the pixels in the white spot area are replaced by symmetrical modification based on the center line, and finally the modified image pixel set is mean-normalized to output the face. However, this patent adjusts the light intensity area based on the threshold, which cannot adapt to the influence of different lighting conditions. The algorithm is not stable enough for the de-reflection effect, and the recognition effect is often deteriorated due to the inaccurate threshold setting.

[0004] Therefore, it is necessary to design a new method to achieve ultra-low light face detection without threshold adjustment, which can perform face detection in different degrees of low light with high accuracy. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a low-light illumination face detection method, device, computer equipment and storage medium.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a low-light illumination face detection method, comprising:

[0007] Obtain the image to be detected;

[0008] Inputting the image to be detected into a face detection model to perform face detection to obtain a detection result;

[0009] outputting the detection result;

[0010] The face detection model is obtained by training a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets.

[0011] A further technical solution is that the face detection model is obtained by training a model consisting of a gradient-guided generative adversarial network and a target detection network using a plurality of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets, including:

[0012] Acquire a number of image pairs consisting of low-light illumination face images and normal-light illumination face images to obtain a sample set;

[0013] Inputting the sample set into a gradient-guided generative adversarial network for processing to obtain a processing result;

[0014] Inputting the processing result into the target detection network to detect the face target frame to obtain the target frame detection result;

[0015] The parameters of the gradient-guided generative adversarial network and the target detection network are adjusted according to the processing results, the target frame detection results and the corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, the gradient-guided generative adversarial network and the target detection network are determined to be face detection models.

[0016] A further technical solution is: inputting the sample set into a gradient-guided generative adversarial network for processing to obtain a processing result, including:

[0017] Inputting the sample set into a gradient-guided generative adversarial network, and processing the low-light illumination face images in the sample set through a Laplacian operator, a Sobel operator, and a Scharr operator to obtain three gradient maps;

[0018] Processing the three gradient maps to obtain three gradient features;

[0019] performing feature extraction on the low-light illumination face images in the sample set to obtain image features;

[0020] Multiplying the image features and the three gradient features in pairs at corresponding positions to obtain three multiplied features;

[0021] The third multiplied feature is processed by the downsampled residual block in the gradient-guided adversarial generative network to obtain three downsampled features;

[0022] Upsampling the third downsampled feature to obtain a feature map and an enhanced face illumination image;

[0023] The third multiplied feature is upsampled, and the upsampled result is directly concatenated with the original feature of the third downsampled feature to obtain the processed result.

[0024] A further technical solution is: extracting features from the low-light illumination facial images in the sample set to obtain image features, including:

[0025] Using the spatial information extraction branch of the gradient-guided generative adversarial network to extract features from the low-light illumination face images in the sample set to obtain spatial features;

[0026] Using a semantic information extraction branch in a gradient-guided generative adversarial network to perform feature extraction on the low-light illumination face images in the sample set to obtain semantic features;

[0027] The spatial features and semantic features are combined to obtain image features.

[0028] Its further technical solution is: the spatial information extraction branch includes spatial separable convolution, SEInception, deformable convolution, two convolutions and relu activation function.

[0029] Its further technical solution is: the semantic information extraction branch includes depth-separable convolution, CBAM module, void convolution, two convolutions and mish activation function.

[0030] A further technical solution is: inputting the processing result into a target detection network to detect a face target frame to obtain a target frame detection result, including:

[0031] The processing result is input into the third downsampling module of the feature extraction hourglass module in the CenterNet network, and the first two downsampling modules in the CenterNet network are deleted to detect the face target frame to obtain the target frame detection result.

[0032] The present invention also provides a low-light illumination face detection device, comprising:

[0033] An image acquisition unit, used for acquiring an image to be detected;

[0034] A detection unit, configured to input the image to be detected into a face detection model to perform face detection to obtain a detection result;

[0035] an output unit, configured to output the detection result;

[0036] It also includes: a model generation unit, which is used to train a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets to obtain a face detection model.

[0037] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0038] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0039] Compared with the prior art, the present invention has the following beneficial effects: the present invention performs face detection on the acquired image to be detected. During the face detection process, a gradient-guided adversarial generative network is used to directly fill in the light for the image whose illumination does not meet the requirements, to generate a light-enhanced face image, and then perform face detection, thereby realizing ultra-low light face detection without threshold adjustment, and can perform face detection for different degrees of low light illumination with high accuracy.

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of an application scenario of the low-light illumination face detection method provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the process of low-light face detection method provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of sub-processes of a low-light illumination face detection method provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of sub-processes of a low-light illumination face detection method provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of sub-processes of a low-light illumination face detection method provided by an embodiment of the present invention;

[0047] Figure 6 A schematic block diagram of a low-light illumination face detection device provided by an embodiment of the present invention;

[0048] Figure 7 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0051] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0052] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0053] See also Figure 1 and Figure 2 , Figure 1 Schematic diagram of an application scenario of the low-light illumination face detection method provided by an embodiment of the present invention. Figure 2 This is a schematic flow chart of a low-light face detection method provided by an embodiment of the present invention. The low-light face detection method is applied to a server. The server exchanges data with a camera and a terminal, captures an image to be detected through the camera, and inputs the image to be detected into a face detection model for face detection. The face detection model directly fills the low-light image with light based on a generative adversarial network to generate a face image with enhanced light, then performs face detection on the image, and outputs the detection result to the terminal for display. This enables ultra-low-light face detection without threshold adjustment, and can perform face detection for different levels of low-light illumination.

[0054] Figure 2 FIG. 1 is a flow chart of a low-light illumination face detection method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S130.

[0055] S110: Acquire an image to be detected.

[0056] In this embodiment, the image to be detected refers to an image captured by a camera and requiring face detection.

[0057] The image to be detected is collected based on the optical camera, and the RGB three-channel image is collected.

[0058] S120: Input the image to be detected into a face detection model to perform face detection to obtain a detection result.

[0059] In this embodiment, the detection result includes the position information of the face target frame if a face exists in the image to be detected; if no face exists in the image to be detected, the detection result indicates that no face exists.

[0060] The face detection model is obtained by training a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets.

[0061] In this embodiment, gradient-guided ultra-low-light face enhancement is based on a gradient-guided deep learning network. This gradient-guided generative adversarial network can effectively enhance low-light images without manual parameter configuration. Furthermore, it can adapt to different angles and degrees of low light, enhancing low-light images.

[0062] In one embodiment, see Figure 3 The above-mentioned face detection model is obtained by training a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light face images and normal-light face images as sample sets, and may include steps S121 to S124.

[0063] S121. Acquire several groups of image pairs consisting of low-light-illumination face images and normal-light-illumination face images to obtain a sample set.

[0064] In this embodiment, an optical camera is used to capture ultra-low-light facial image data, capturing RGB three-channel images. A pair of low-light facial images and a matching normal-light facial image are captured, forming an image pair consisting of a low-light facial image and a normal-light facial image. The sample set consists of several image pairs consisting of low-light facial images and normal-light facial images.

[0065] S122: Input the sample set into a gradient-guided generative adversarial network for processing to obtain a processing result.

[0066] In this embodiment, the processing result refers to a feature map formed after the gradient-guided adversarial generative network performs fill-light processing.

[0067] The gradient-guided generative adversarial network directly fills the low-light image with light to generate an enhanced-light face image.

[0068] In one embodiment, see Figure 4 , the above-mentioned step S122 may include steps S1221 to S1227.

[0069] S1221. Input the sample set into a gradient-guided generative adversarial network, and process the low-light illumination face images in the sample set through a Laplacian operator, a Sobel operator, and a Scharr operator to obtain three gradient maps.

[0070] In this embodiment, the three gradient images refer to the low-light face image I input The image gradient transformation map is obtained by using the Laplacian operator, Sobel operator, and Scharr operator.

[0071] Specifically, the input low-light face image I input Through the Laplacian operator, Sobel operator and Scharr operator, the image gradient transformation map is obtained as I la , I so and I scThree gradient maps, here the kernel sizes are 3, 5, and 5 respectively.

[0072] S1222: Process the three gradient images to obtain three gradient features.

[0073] In this embodiment, the three gradient features refer to the three gradient images I la , I so and I sc The multiple gradient feature maps obtained after processing by the three residual blocks are respectively processed and then upsampled to form features.

[0074] Specifically, for the three gradient images I la , I so and I sc Through three residual blocks, multiple gradient feature maps are obtained, which are F g2 , F g3 , the three residual blocks here are composed of 2 convolutions and 1 activation function. Then the upsampling operation is performed, and the upsampling here is performed by deconvolution to obtain three gradient features F respectively. u1 ,F u2 ,F u3 .

[0075] S1223: Perform feature extraction on the low-light illumination facial images in the sample set to obtain image features.

[0076] In this embodiment, the image feature refers to the low-light illumination face image I input The features formed after feature extraction, here the feature extraction network is divided into two branches, one for spatial information extraction and the other for semantic feature extraction.

[0077] In one embodiment, see Figure 4 , the above-mentioned step S1223 may include steps S12231 to S12233.

[0078] S12231. Use the spatial information extraction branch in the gradient-guided generative adversarial network to perform feature extraction on the low-light illumination face images in the sample set to obtain spatial features.

[0079] In this embodiment, the spatial feature refers to the spatial information of the image.

[0080] Specifically, the spatial information extraction branch includes spatial separable convolution, SE Inception, deformable convolution, two convolutions and relu activation function.

[0081] The spatial information extraction branch performs corresponding feature extraction through multiple spatial information extraction blocks. The spatial information extraction block is composed of spatial separable convolution, SE Inception and deformable convolution, and then connected to two convolutions and relu activation functions to generate feature F. spatial .

[0082] S12232. Use the semantic information extraction branch in the gradient-guided generative adversarial network to perform feature extraction on the low-light illumination face images in the sample set to obtain semantic features.

[0083] In this embodiment, the semantic feature refers to the semantic information of the image.

[0084] The semantic information extraction branch includes depthwise separable convolution, CBAM module, dilated convolution, two convolutions and mish activation function.

[0085] The semantic information extraction branch extracts corresponding features through multiple semantic information extraction blocks. The semantic information extraction block is composed of depthwise separable convolution, CBAM (Convolutional Block Attention Module) and dilated convolution, which are then connected to two convolutions and mish activation functions to generate feature F. depth .

[0086] S12233. Merge the spatial features and semantic features to obtain image features.

[0087] In this embodiment, the features output by each spatial information extraction block and each semantic information extraction block are merged. Here, the number of spatial information extraction blocks is 3 and the number of semantic information extraction blocks is 5. Thus, the image feature F is obtained through three stages of spatial information extraction and semantic information extraction. sd-1 、F sd-2 、F sd-3 .

[0088] S1224: Multiply the elements at corresponding positions of the image features and the three gradient features by two of each other to obtain three multiplied features.

[0089] In this embodiment, the multiplication feature refers to a feature formed by multiplying elements of the image feature and the gradient feature at the corresponding position by two or more.

[0090] Specifically, according to the three gradient feature maps F u1 ,F u2 ,F u3 The corresponding position elements and image features are multiplied by two to obtain three multiplication features F usd1 、F usd2 、F usd3 .

[0091] S1225. Process the third multiplied feature through the downsampled residual block in the gradient-guided adversarial generative network to obtain three downsampled features.

[0092] In this embodiment, the downsampled feature refers to a feature formed by processing the third multiplied feature through a downsampled residual block in a gradient-guided adversarial generative network.

[0093] Specifically, the third multiplication feature F usd3 After processing by three down-sampling residual blocks, three down-sampling features F1, F2, and F3 are obtained respectively.

[0094] S1226. Perform upsampling on the third downsampled feature to obtain a feature map and an enhanced facial illumination image.

[0095] In this embodiment, the feature map refers to the map formed by upsampling the third downsampled feature; the enhanced face illumination image refers to the image with 3 channels and a size of 1. input Face image with enhanced facial lighting.

[0096] Specifically, the third down-sampled feature F3 is up-sampled, and the up-sampling process uses difference, deconvolution and de-pooling respectively to obtain the feature map And the number of channels is 3 and the size is 1 input Face image with enhanced facial lighting

[0097] S1227. Upsample the third multiplied feature, and directly concatenate the upsampled result with the third downsampled feature to obtain a processing result.

[0098] Specifically, the third multiplicative feature F in the gradient-guided adversarial generation network is usd3 After upsampling, the original features are directly spliced ​​with the third downsampled feature F3 to obtain the feature map F E , as the processing result, and input into the target detection network for feature extraction to realize the detection of face target frame.

[0099] Gradient-guided low-light face enhancement is based on a deep learning network guided by gradients. It can effectively enhance low-light images without manual parameter configuration. It can adapt to lighting at different angles and different degrees of low light and perform enhancement operations on low-light images.

[0100] S123: Input the processing result into the target detection network to detect the face target frame to obtain the target frame detection result.

[0101] In this embodiment, the target frame detection result refers to the detection result of the position of the face target frame and the like.

[0102] Specifically, the processing result is input into the third downsampling module of the feature extraction hourglass module in the CenterNet network, and the first two downsampling modules in the CenterNet network are deleted to detect the face target frame to obtain the target frame detection result.

[0103] The target detection network uses the anchor-free CenterNet; the input of the feature extraction network here uses the F in the gradient-guided adversarial generation network. usd3 The feature map F is obtained by concatenating the upsampling and F3 E The data is input into the third downsampling module of the feature extraction hourglass module of CenterNet, and the first two downsampling modules are deleted. The coordinate information of the face can be well obtained through the prediction of the network.

[0104] S124. Adjust the parameters of the gradient-guided generative adversarial network and the target detection network according to the processing results, the target frame detection results and the corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, determine that the gradient-guided generative adversarial network and the target detection network are face detection models.

[0105] In this embodiment, the loss value is calculated based on the processing results, the target frame detection results and the corresponding loss function. Depending on whether the loss value tends to converge, if it does not tend to converge, the parameters of the gradient-guided generative adversarial network and the target detection network are adjusted and retrained until the loss value converges and the model training converges. Only then are the final gradient-guided generative adversarial network and the target detection network combined to form a face detection model.

[0106] In this embodiment, the loss function of the gradient-guided adversarial generative network, i.e., the adversarial loss function, includes an FSIM loss function, a pixel loss function, a perceptual loss function, and a gradient loss function. The pixel loss function uses the MSE loss function: Here, α is set to 0.6, and B in the formula represents a face image under normal lighting conditions. It represents the generated face enhanced illumination image. The perceptual loss function is: The gradient loss function is Here It represents the gradient of the image itself under normal lighting conditions. It represents the gradient of the image itself under the generated enhanced lighting conditions. The FSIM loss function is: The adversarial loss function is: Loss total =αLoss pixel +βLoss perceptual +γLoss gradient +δLoss adversarial ; Here, α, β, γ, and δ are 0.3, 0.2, 0.2, and 0.3 respectively.

[0107] The loss functions of the target detection network include the target center point loss function, the target size loss function, and the target center point bias loss function.

[0108] S130: Output the detection result.

[0109] The method of this embodiment does not require excessive manual parameter adjustment for facial image illumination enhancement. The model also supports image illumination enhancement for different low-light intensities and directions. Furthermore, the image illumination enhancement and target detection models can be combined into a single face detection model, effectively performing face detection under low-light conditions. This results in higher efficiency and better results for face detection under low-light conditions.

[0110] The above-mentioned low-light face detection method performs face detection on the acquired image to be detected. During the face detection process, a gradient-guided adversarial generative network is used to directly fill in the light for images whose illumination does not meet the requirements, generate a face image with enhanced light, and then perform face detection. This realizes ultra-low-light face detection without threshold adjustment, and can perform face detection for different degrees of low-light illumination with high accuracy.

[0111] Figure 6 FIG is a schematic block diagram of a low-light illumination face detection device 300 provided by an embodiment of the present invention. Figure 6 As shown, corresponding to the above low-light illumination face detection method, the present invention also provides a low-light illumination face detection device 300. The low-light illumination face detection device 300 includes a unit for executing the above low-light illumination face detection method, and the device can be configured in a server. Figure 6 The low-light illumination face detection device 300 includes an image acquisition unit 301, a detection unit 302 and an output unit 303.

[0112] The image acquisition unit 301 is used to acquire an image to be detected; the detection unit 302 is used to input the image to be detected into a face detection model to perform face detection to obtain a detection result; and the output unit 303 is used to output the detection result.

[0113] In one embodiment, the low-light face detection device 300 also includes a model generation unit, which is used to train a model composed of a gradient-guided adversarial generative network and a target detection network using several groups of image pairs consisting of low-light face images and normal-light face images as sample sets to obtain a face detection model.

[0114] In one embodiment, the model generation unit includes a sample set acquisition subunit, a processing subunit, a target frame detection subunit, and an adjustment subunit.

[0115] The sample set acquisition subunit is used to obtain several groups of image pairs consisting of low-light face images and normal-light face images to obtain a sample set; the processing subunit is used to input the sample set into the gradient-guided adversarial generation network for processing to obtain a processing result; the target frame detection subunit is used to input the processing result into the target detection network to detect the face target frame to obtain a target frame detection result; the adjustment subunit is used to adjust the parameters of the gradient-guided adversarial generation network and the target detection network according to the processing result, the target frame detection result and the corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, determine that the gradient-guided adversarial generation network and the target detection network are face detection models.

[0116] In one embodiment, the processing subunit includes a gradient processing module, a gradient map processing module, a feature extraction module, a multiplication module, a downsampling processing module, an upsampling processing module, and a splicing processing module.

[0117] A gradient processing module is used to input the sample set into the gradient-guided adversarial generative network, and the low-light illumination face image in the sample set is processed by the Laplacian operator, the Sobel operator and the Scharr operator to obtain three gradient maps; a gradient map processing module is used to process the three gradient maps to obtain three gradient features; a feature extraction module is used to extract features from the low-light illumination face image in the sample set to obtain image features; a multiplication module is used to multiply the image features and the three gradient features by two-by-two elements at corresponding positions to obtain three multiplied features; a downsampling processing module is used to process the third multiplied feature through the downsampling residual block in the gradient-guided adversarial generative network to obtain three downsampling features; an upsampling processing module is used to upsample the third downsampling feature to obtain a feature map and an enhanced face illumination image; a splicing processing module is used to upsample the third multiplied feature, and directly splice the upsampling result with the original feature of the third downsampling feature to obtain a processing result.

[0118] In one embodiment, the feature extraction module includes a spatial extraction submodule, a semantic extraction submodule, and a feature merging submodule.

[0119] The spatial extraction submodule is used to use the spatial information extraction branch in the gradient-guided adversarial generative network to perform feature extraction on the low-light illumination face images in the sample set to obtain spatial features; the semantic extraction submodule is used to use the semantic information extraction branch in the gradient-guided adversarial generative network to perform feature extraction on the low-light illumination face images in the sample set to obtain semantic features; and the feature merging submodule is used to merge the spatial features and semantic features to obtain image features.

[0120] In one embodiment, the target frame detection subunit is used to input the processing result into the third downsampling module of the feature extraction hourglass module in the CenterNet network, delete the first two downsampling modules in the CenterNet network, and perform face target frame detection to obtain a target frame detection result.

[0121] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned low-light illumination face detection device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment. For the convenience and brevity of the description, it will not be repeated here.

[0122] The low-light illumination face detection device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 7 Runs on the computer equipment shown.

[0123] See also Figure 7 , Figure 7 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0124] See Figure 7 The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0125] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to perform a low-light illumination face detection method.

[0126] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0127] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can perform a low-light illumination face detection method.

[0128] The network interface 505 is used to communicate with other devices through the network. Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0129] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0130] Acquire an image to be detected; input the image to be detected into a face detection model to perform face detection to obtain a detection result; and output the detection result;

[0131] The face detection model is obtained by training a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets.

[0132] In one embodiment, when the processor 502 implements the step of training a model consisting of a gradient-guided generative adversarial network and an object detection network using a plurality of sets of image pairs consisting of low-light face images and normal-light face images as sample sets, the processor 502 specifically implements the following steps:

[0133] Several groups of image pairs consisting of low-light facial images and normal-light facial images are obtained to obtain a sample set; the sample set is input into a gradient-guided generative adversarial network for processing to obtain a processing result; the processing result is input into a target detection network to detect a facial target frame to obtain a target frame detection result; parameters of the gradient-guided generative adversarial network and the target detection network are adjusted according to the processing result and the target frame detection result in combination with a corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, the gradient-guided generative adversarial network and the target detection network are determined to be a face detection model.

[0134] In one embodiment, when the processor 502 performs processing within the sample set input gradient-guided generative adversarial network to obtain a processing result, the processor 502 specifically implements the following steps:

[0135] The sample set is input into a gradient-guided generative adversarial network, and the low-light illumination face images in the sample set are processed by the Laplacian operator, the Sobel operator and the Scharr operator to obtain three gradient maps; the three gradient maps are processed to obtain three gradient features; feature extraction is performed on the low-light illumination face images in the sample set to obtain image features; the image features and the three gradient features are multiplied in pairs at corresponding position elements to obtain three multiplied features; the third multiplied feature is processed by the down-sampled residual block in the gradient-guided generative adversarial network to obtain three down-sampled features; the third down-sampled feature is up-sampled to obtain a feature map and an enhanced face illumination image; the third multiplied feature is up-sampled, and the up-sampled result is directly spliced ​​with the original feature of the third down-sampled feature to obtain a processing result.

[0136] In one embodiment, when the processor 502 implements the step of extracting features from the low-light illumination facial images in the sample set to obtain image features, the processor 502 specifically implements the following steps:

[0137] The spatial information extraction branch in the gradient-guided generative adversarial network is used to extract features of the low-light facial images in the sample set to obtain spatial features; the semantic information extraction branch in the gradient-guided generative adversarial network is used to extract features of the low-light facial images in the sample set to obtain semantic features; the spatial features and the semantic features are merged to obtain image features.

[0138] Among them, the spatial information extraction branch includes spatial separable convolution, SE Inception, deformable convolution, two convolutions and relu activation function.

[0139] The semantic information extraction branch includes depthwise separable convolution, CBAM module, dilated convolution, two convolutions and mish activation function.

[0140] In one embodiment, when the processor 502 implements the step of inputting the processing result into the target detection network to detect the face target frame to obtain the target frame detection result, it specifically implements the following steps:

[0141] The processing result is input into the third downsampling module of the feature extraction hourglass module in the CenterNet network, and the first two downsampling modules in the CenterNet network are deleted to detect the face target frame to obtain the target frame detection result.

[0142] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0144] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0145] Acquire an image to be detected; input the image to be detected into a face detection model to perform face detection to obtain a detection result; and output the detection result;

[0146] The face detection model is obtained by training a model composed of a gradient-guided adversarial generation network and a target detection network using several groups of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets.

[0147] In one embodiment, when the processor executes the computer program to implement the step of training a model consisting of a gradient-guided generative adversarial network and an object detection network using a plurality of sets of image pairs consisting of low-light facial images and normal-light facial images as sample sets, the processor specifically implements the following steps:

[0148] Several groups of image pairs consisting of low-light facial images and normal-light facial images are obtained to obtain a sample set; the sample set is input into a gradient-guided generative adversarial network for processing to obtain a processing result; the processing result is input into a target detection network to detect a facial target frame to obtain a target frame detection result; parameters of the gradient-guided generative adversarial network and the target detection network are adjusted according to the processing result and the target frame detection result in combination with a corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, the gradient-guided generative adversarial network and the target detection network are determined to be a face detection model.

[0149] In one embodiment, when the processor executes the computer program to implement the step of inputting the sample set into the gradient-guided generative adversarial network for processing to obtain a processing result, the processor specifically implements the following steps:

[0150] The sample set is input into a gradient-guided generative adversarial network, and the low-light illumination face images in the sample set are processed by the Laplacian operator, the Sobel operator and the Scharr operator to obtain three gradient maps; the three gradient maps are processed to obtain three gradient features; feature extraction is performed on the low-light illumination face images in the sample set to obtain image features; the image features and the three gradient features are multiplied in pairs at corresponding position elements to obtain three multiplied features; the third multiplied feature is processed by the down-sampled residual block in the gradient-guided generative adversarial network to obtain three down-sampled features; the third down-sampled feature is up-sampled to obtain a feature map and an enhanced face illumination image; the third multiplied feature is up-sampled, and the up-sampled result is directly spliced ​​with the original feature of the third down-sampled feature to obtain a processing result.

[0151] In one embodiment, when the processor executes the computer program to implement the step of extracting features from the low-light illumination facial images in the sample set to obtain image features, the processor specifically implements the following steps:

[0152] The spatial information extraction branch in the gradient-guided generative adversarial network is used to extract features of the low-light facial images in the sample set to obtain spatial features; the semantic information extraction branch in the gradient-guided generative adversarial network is used to extract features of the low-light facial images in the sample set to obtain semantic features; the spatial features and the semantic features are merged to obtain image features.

[0153] Among them, the spatial information extraction branch includes spatial separable convolution, SE Inception, deformable convolution, two convolutions and relu activation function.

[0154] The semantic information extraction branch includes depthwise separable convolution, CBAM module, dilated convolution, two convolutions and mish activation function.

[0155] In one embodiment, when the processor executes the computer program to implement the step of inputting the processing result into the target detection network to detect the face target frame to obtain the target frame detection result, the processor specifically implements the following steps:

[0156] The processing result is input into the third downsampling module of the feature extraction hourglass module in the CenterNet network, and the first two downsampling modules in the CenterNet network are deleted to detect the face target frame to obtain the target frame detection result.

[0157] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0158] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0159] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0160] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0161] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A low-light illumination face detection method, characterized in that: include: Obtain the image to be detected; Inputting the image to be detected into a face detection model to perform face detection to obtain a detection result; outputting the detection result; The face detection model is obtained by training a model composed of a gradient-guided generative adversarial network and a target detection network using a plurality of sets of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets; The face detection model is obtained by training a model composed of a gradient-guided adversarial generative network and a target detection network using several sets of image pairs consisting of low-light illumination face images and normal-light illumination face images as sample sets, including: Acquire a number of image pairs consisting of low-light illumination face images and normal-light illumination face images to obtain a sample set; Inputting the sample set into a gradient-guided generative adversarial network for processing to obtain a processing result; Inputting the processing result into the target detection network to detect the face target frame to obtain the target frame detection result; Adjusting parameters of a gradient-guided generative adversarial network and a target detection network according to the processing results and the target frame detection results in combination with a corresponding loss function, and determining that the gradient-guided generative adversarial network and the target detection network are face detection models when loss values ​​calculated by the corresponding loss functions tend to converge; Inputting the sample set into the gradient-guided generative adversarial network for processing to obtain a processing result includes: Inputting the sample set into a gradient-guided generative adversarial network, and processing the low-light illumination face images in the sample set through a Laplacian operator, a Sobel operator, and a Scharr operator to obtain three gradient maps; Processing the three gradient maps to obtain three gradient features; performing feature extraction on the low-light illumination face images in the sample set to obtain image features; Multiplying the image features and the three gradient features in pairs at corresponding positions to obtain three multiplied features; The third multiplied feature is processed by the downsampled residual block in the gradient-guided adversarial generative network to obtain three downsampled features; Upsampling the third downsampled feature to obtain a feature map and an enhanced face illumination image; The third multiplied feature is upsampled, and the upsampled result is directly concatenated with the original feature of the third downsampled feature to obtain the processed result.

2. The low-light illumination face detection method according to claim 1, characterized in that: The extracting features of the low-light illumination face images in the sample set to obtain image features includes: Using the spatial information extraction branch of the gradient-guided generative adversarial network to extract features from the low-light illumination face images in the sample set to obtain spatial features; Using a semantic information extraction branch in a gradient-guided generative adversarial network to perform feature extraction on the low-light illumination face images in the sample set to obtain semantic features; The spatial features and semantic features are combined to obtain image features.

3. The low-light illumination face detection method according to claim 2, characterized in that: The spatial information extraction branch includes spatial separable convolution, SE Inception, deformable convolution, two convolutions and relu activation function.

4. The low-light illumination face detection method according to claim 2, characterized in that: The semantic information extraction branch includes depthwise separable convolution, CBAM module, dilated convolution, two convolutions and mish activation function.

5. The low-light illumination face detection method according to claim 1, characterized in that: Inputting the processing result into the target detection network to detect the face target frame to obtain the target frame detection result includes: The processing result is input into the third downsampling module of the feature extraction hourglass module in the CenterNet network, and the first two downsampling modules in the CenterNet network are deleted to detect the face target frame to obtain the target frame detection result.

6. Low-light illumination face detection device, characterized in that: include: An image acquisition unit, used for acquiring an image to be detected; A detection unit, configured to input the image to be detected into a face detection model to perform face detection to obtain a detection result; an output unit, configured to output the detection result; The system further includes: a model generation unit for training a model composed of a gradient-guided generative adversarial network and a target detection network using a plurality of image pairs composed of low-light illumination face images and normal-light illumination face images as sample sets to obtain a face detection model; The model generation unit includes a sample set acquisition subunit, a processing subunit, a target frame detection subunit and an adjustment subunit; A sample set acquisition subunit is configured to acquire a plurality of image pairs consisting of low-light illumination face images and normal-light illumination face images to obtain a sample set; a processing subunit is configured to input the sample set into a gradient-guided generative adversarial network for processing to obtain a processing result; a target frame detection subunit is configured to input the processing result into a target detection network for detecting a face target frame to obtain a target frame detection result; an adjustment subunit is configured to adjust parameters of the gradient-guided generative adversarial network and the target detection network based on the processing result and the target frame detection result in combination with a corresponding loss function, and when the loss value calculated by the corresponding loss function tends to converge, determine that the gradient-guided generative adversarial network and the target detection network are a face detection model; The processing subunit includes a gradient processing module, a gradient map processing module, a feature extraction module, a multiplication module, a downsampling processing module, an upsampling processing module and a splicing processing module; A gradient processing module is used to input the sample set into the gradient-guided adversarial generative network, and the low-light illumination face image in the sample set is processed by the Laplacian operator, the Sobel operator and the Scharr operator to obtain three gradient maps; a gradient map processing module is used to process the three gradient maps to obtain three gradient features; a feature extraction module is used to extract features from the low-light illumination face image in the sample set to obtain image features; a multiplication module is used to multiply the image features and the three gradient features by two-by-two elements at corresponding positions to obtain three multiplied features; a downsampling processing module is used to process the third multiplied feature through the downsampling residual block in the gradient-guided adversarial generative network to obtain three downsampling features; an upsampling processing module is used to upsample the third downsampling feature to obtain a feature map and an enhanced face illumination image; a splicing processing module is used to upsample the third multiplied feature, and directly splice the upsampling result with the original feature of the third downsampling feature to obtain a processing result.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images