Face image enhancement and recognition method in low-light environment

Abstract

The application discloses a face image enhancement and recognition method in a low-light environment, and relates to the technical field of face recognition; first, a low-light face image is subjected to multi-scale Retinex enhancement to improve brightness and details; then, LBP and HOG features of the enhanced image are extracted, and the features are dynamically adjusted in weight and fused according to light; then, a residual attention module is used to perform spatial and channel weighting optimization on the fused features; finally, the optimized features are input into a pre-trained model to extract vectors, and cosine similarity matching recognition is performed to complete face processing and identity determination in a low-light environment. The application solves the problem of low-light face recognition, improves image quality through multi-scale enhancement, dynamically fuses features, and strengthens feature extraction through an attention mechanism to improve recognition accuracy and stability; no additional light supplement is needed, cost is reduced, and live detection is added to ensure safety, and the application is suitable for multiple scenes such as security and finance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a face image enhancement and recognition method in a low-light environment. BACKGROUND

[0002] In many fields such as security monitoring, access control management, mobile payment, face recognition technology has become an important means of identity verification. However, in low-light environments such as night, underground parking lot, dim indoor environment, the performance of traditional face recognition systems will decrease significantly. Low light causes uneven brightness, blurred details, increased noise, and other problems in face images, making it difficult to extract facial features in the image. Existing technologies usually only use a single image enhancement method, such as histogram equalization, but this method can easily over-enhance noise, leading to image distortion and failing to effectively improve recognition results. In addition, some systems rely on light supplement devices to improve image quality, which not only increases hardware costs and deployment difficulty, but also may cause user discomfort due to light interference, and cannot meet the use requirements in complex scenarios.

[0003] With the wide application of deep learning in the field of computer vision, some face recognition algorithms based on convolutional neural networks (CNN) are used in low-light scenarios. However, these algorithms often directly extract features from original low-quality images, making it difficult to overcome the impact of uneven lighting, resulting in inaccurate feature extraction. For example, when the light is too dark, the network model may not be able to accurately capture the detailed features of key parts such as eyes and nose, resulting in a significant reduction in recognition accuracy. At the same time, the existing image enhancement and recognition process is independent of each other, and lacks targeted and collaborative optimization for low-light environments. The enhanced image may not match the input requirements of the recognition model, further affecting the overall performance.

[0004] Currently, although some research attempts to combine image enhancement and recognition, there are still deficiencies in feature processing. Traditional methods often use fixed weights when fusing different types of image features, which cannot adapt to the dynamic changes of image features in low-light environments. For example, under extremely low light conditions, the importance of local texture features may increase significantly, but the fixed weight fusion method cannot highlight these key features, resulting in insufficient robustness of the algorithm. In addition, existing technologies lack effective comprehensive processing mechanisms when facing complex noise and uneven lighting mixed interference, making it difficult to improve image quality while ensuring recognition accuracy and stability. SUMMARY

[0005] The face image enhancement and recognition method in a low-light environment proposed by the present application solves the problems mentioned in the above existing technologies.

[0006] In order to achieve the above object, the present application adopts the following technical scheme: A face image enhancement and recognition method in low light environment, comprising the following steps:

[0007] Multi-scale Retinex enhancement step: three-scale Gaussian filtering is performed on the input low-light face image, and the scale parameters of the three scales are set to 15, 80 and 250 respectively, and the illumination component and the reflection component under different scales are separated by filtering; the adaptive segmented logarithmic transformation function is used for nonlinear enhancement of the reflection component, and the threshold and compensation parameters are dynamically adjusted according to the global brightness mean value of the image, so that the noise amplification is suppressed while the dark details are improved;

[0008] Adaptive feature fusion step: local binary pattern (LBP) feature and histogram of oriented gradient (HOG) feature of the enhanced image are extracted, the illumination condition is evaluated by calculating the global brightness mean value of the image, and the fusion weight of the two types of features is dynamically adjusted: the weight proportion of the LBP texture feature is automatically increased when the illumination is weak, and the fine texture of the face is highlighted; the weight of the HOG gradient feature is increased when the illumination is strong, and the contour information is strengthened; the feature vector is generated through the dynamic fusion strategy;

[0009] Attention mechanism optimization step: a residual attention module is constructed, spatial features are extracted through convolution operation and a spatial attention weight matrix is generated, and the face region is focused; at the same time, the global average pooling and the full connection layer are used to calculate the channel attention weight matrix, and the interference of irrelevant background channels is suppressed; the spatial and channel attention weight matrices are multiplied element by element, and the fusion features are weighted and optimized to improve the feature expression ability of the region, and the discriminative feature vector is output.

[0010] Further, it further comprises:

[0011] Illumination evaluation preprocessing step: the global brightness mean value of the input image is calculated, and when the mean value is lower than the set threshold, the enhancement process is triggered, and the scale parameters of the multi-scale Retinex enhancement are dynamically adjusted according to the brightness mean value, so as to meet the image enhancement requirements under different illumination intensities.

[0012] Noise suppression preprocessing step: non-local mean filtering algorithm is used for denoising processing of the input image, the similarity weight between the current pixel and the pixels in the search window is calculated, the noise is suppressed while the image edge details are preserved, and a clear image after denoising is generated.

[0013] Further, in the multi-scale Retinex enhancement step, when the reflection components of different scales are weighted and fused, the weight is dynamically adjusted according to the local texture features of the image, and the scale component corresponding to the region of the texture is given higher weight.

[0014] Further, it further comprises:

[0015] Adaptive histogram equalization step: convert the enhanced image to YCbCr color space, block local histogram equalization on the luminance component to improve local contrast; eliminate the blocking effect of sub-region boundaries by bilinear interpolation.

[0016] Further, in the attention mechanism optimization step, the spatial attention module extracts spatial features through convolution operation, the channel attention module calculates channel weights through global average pooling and fully connected layer, and finally generates an attention weight matrix through element-wise multiplication.

[0017] Further, it further comprises:

[0018] Face recognition matching step: input the optimized feature vector into the pre-trained face recognition model to extract the face feature vector and calculate the cosine similarity with the registered vector in the database, and determine whether the face recognition is successful according to the similarity value.

[0019] Quality evaluation post-processing step: calculate the sharpness index of the enhanced image, and evaluate the image quality based on the gradient change in the horizontal and vertical directions; when the sharpness is lower than the threshold, adjust the enhancement parameters through iteration to meet the quality requirements.

[0020] Liveness detection auxiliary step: extract the local ternary pattern LTP texture feature and the optical flow field micro-motion feature of the enhanced image, and input them into the liveness detection model for true or false judgment to resist photo and video non-liveness attacks.

[0021] Further, the segmented logarithmic transformation function used in the multi-scale Retinex enhancement step dynamically adjusts the enhancement parameters according to the image brightness, realizing the processing of different illumination regions.

[0022] Compared with the existing technology, the beneficial effects of the present application are:

[0023] Through multi-technology fusion and innovation strategy, the accuracy and stability of face recognition in low light environment are improved. In the image enhancement stage, multi-scale Retinex enhancement and adaptive histogram equalization technology are combined to accurately adjust the brightness and contrast of the image, effectively avoid noise amplification and detail loss, and make the enhanced image clearer and more natural, providing high-quality data basis for subsequent recognition.

[0024] Adaptive feature fusion and attention mechanism optimization are the core advantages of the method. According to the illumination condition, the fusion weights of LBP and HOG features are dynamically adjusted, which can fully play the advantages of each feature under different illumination intensities, and ensure that the key features are accurately extracted. The residual attention module further focuses on the key areas of the face, suppresses background interference, enhances the expression ability of the features, and greatly improves the processing ability of the model for low-quality images.

[0025] In practical applications, the method improves the accuracy and efficiency of face recognition. Compared with traditional methods, even in extremely dark environments, the method can still maintain a high recognition success rate, reduce false recognition and rejection caused by lighting problems. At the same time, through the pre-processing and post-processing steps of light evaluation, noise suppression and quality evaluation, a complete closed-loop optimization system is formed, which effectively deals with complex environmental interference, reduces the dependence on external light equipment, reduces deployment costs and use restrictions. In addition, the living body detection auxiliary step ensures the safety of the identification process, and provides a reliable technical solution for fields such as security and finance that require high accuracy and safety. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A schematic block diagram of the face image enhancement and recognition method in low-light environment proposed by the present application;

[0027] Figure 2 A schematic diagram of the clarity comparison of the image enhancement of different methods of the face image enhancement and recognition in low-light environment proposed by the present application;

[0028] Figure 3 A schematic diagram of the multi-scale Retinex enhancement effect comparison of different methods of the face image enhancement and recognition in low-light environment proposed by the present application;

[0029] Figure 4 A schematic diagram of the adaptive feature fusion weight dynamic adjustment of the face image enhancement and recognition in low-light environment proposed by the present application DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] Reference Figure 1 and Figure 4 : A specific embodiment of a face image enhancement and recognition method in low-light environment.

[0032] The face image enhancement and recognition method in low-light environment of the present application realizes high-quality enhancement and accurate recognition of face images in low-light scenes through multi-step cooperative processing. The specific implementation details are as follows:

[0033] I. Multi-scale Retinex enhancement step

[0034] 1. Three-scale Gaussian filtering and component separation technology details

[0035] Perform three-scale Gaussian filtering on the input low-light face image I(x,y) (resolution (640x480)). The specific implementation is as follows:

[0036] Small scale filtering ( =15): Uses a 31x31 pixel Gaussian kernel with a standard deviation of 15 to capture high-frequency details such as eyelashes and pores while suppressing local noise. High-Frequency Detail Capture: σ = 15 corresponds to a smaller Gaussian kernel (31×31) with a limited filtering range. It is primarily used to extract high-frequency details in facial images (such as eyelashes, pores, and wrinkles) while suppressing local noise.

[0037] Balancing computational efficiency and detail preservation: The Gaussian kernel size is generally positively correlated with σ (the theoretical kernel size is 6σ + 1, or 15 × 6 + 1 = 91). A 31 × 31 kernel is used to reduce computational effort while ensuring detail extraction. Experiments have shown that this size effectively preserves subtle facial textures and significantly suppresses noise.

[0038] Mesoscale filtering ( =80): Use a Gaussian kernel of 161x161 pixels with a standard deviation of 80 to extract facial organ contours (such as eyes and nose boundaries) and medium-scale structures.

[0039] Large-scale filtering ( =250): The large-scale filter uses a Gaussian kernel size of 501×501 pixels with a standard deviation of 250. It is mainly used to show the overall lighting conditions of the image and identify areas with uneven brightness, such as large shadows. Next, we need to separate the illumination component and the reflection component of the image. This is achieved through convolution. The illumination component L i (x,y) is obtained as follows: take the original image I(x,y) and a two-dimensional Gaussian kernel function The result of the convolution operation is the illumination component. i The calculation of (x,y) is a little more complicated. First, take the natural logarithm of the original image I(x,y), and then subtract the illumination component L i The natural logarithm of (x, y) is used to convert the image from the original pixel domain to the logarithmic domain, so that we can distinguish between the illumination and reflection components.

[0040] Global illumination trend characterization: σ = 250 corresponds to a large-scale Gaussian kernel (501 × 501), whose coverage can capture the global illumination changes in the image (such as large shadows and uneven light and dark areas) and separate the illumination components through convolution operations.

[0041] Indirect verification of light uniformity: After large-scale filtering, the enhancement parameters of dark and bright parts are dynamically adjusted by adaptive piecewise logarithmic transformation function, combined with texture adaptive weighted fusion strategy, which can effectively improve the dark details and suppress noise amplification. Experimental data show that the recognition accuracy is improved in low light environment (≤5 lux), which indirectly proves the effectiveness of large-scale parameters in light uniformity processing.

[0042] Single-scale filtering cannot simultaneously consider detail preservation and global light adjustment (such as single-scale Retinex enhancement, which can cause dark noise amplification or bright overexposure), while three scales (15, 80, 250) correspond to high-frequency details, medium structures, and global light, forming a hierarchical processing flow.

[0043] The calculation of large-scale kernel (501x501) is large, but the invention optimizes the processing delay to 85ms / frame through lightweight design (such as depth separable convolution), which meets the real-time requirement. This shows that the parameter setting is a trade-off between light processing effect and hardware efficiency. The coverage range of 501x501 kernel is verified by experiments to ensure the accuracy of light analysis while avoiding excessive consumption of computing resources.

[0044] The setting of three-scale parameters (15, 80, 250) is based on the hierarchical feature extraction requirements (high-frequency details→medium structures→global light) and experimental optimization results, which improves the enhancement effect of low-light images through multi-scale collaborative processing.

[0045] When processing low-light face images, three-scale Gaussian filtering and component separation technology play a key role. First, for the input low-light face image I(x,y) with a resolution of 640x480, three-scale Gaussian filtering operation is performed.

[0046] Small-scale filtering uses a 31x31 pixel Gaussian kernel with a standard deviation of 15. This scale of filtering aims to capture high-frequency details in the face image, such as eyelashes, pores, and other subtle details. At the same time, it can effectively suppress local noise, making these high-frequency details clearer and more prominent, avoiding blurring due to noise interference.

[0047] Medium-scale filtering uses a 161x161 pixel Gaussian kernel with a standard deviation of 80. This scale focuses on extracting contour information of facial organs, such as the shape boundary of the eyes and the contour of the nose wing. These medium-scale structures can be accurately captured, which is crucial for accurately identifying facial features and laying the foundation for subsequent analysis and processing.

[0048] The large-scale filter uses a 501x501 pixel Gaussian kernel with a standard deviation of 250. It is mainly used to characterize the global illumination trend of the image, and can sensitively identify the unevenly bright and dark areas in the image, such as large areas of shadow. By grasping the global illumination, it helps to understand the overall illumination distribution of the image, and provides an important basis for subsequent illumination adjustment and image enhancement.

[0049] After completing the three-scale Gaussian filtering, the illumination component Li(x, y) and the reflection component Ri(x, y) are separated by convolution operation. Specifically, the illumination component Li(x, y) is obtained by convolution operation of the image I(x, y) and the two-dimensional Gaussian kernel function G(x, y; σi), that is, Li(x, y) = I(x, y) * G(x, y; σi). And the reflection component Ri(x, y) is calculated by the formula Ri(x, y) = ln(I(x, y)) - ln(Li(x, y)) by converting the image from the pixel domain to the logarithmic domain. Here, ln represents the natural logarithm operation, and in this way, the illumination and reflection information can be effectively separated, providing a clear and accurate image component basis for further processing of low-light face images, improving image quality, enhancing details, and other subsequent operations.

[0050] 2. Adaptive piecewise logarithmic transformation optimization

[0051] When processing the image reflection component, we designed a dynamic enhancement function, the core parameters of which will be automatically adjusted according to the global brightness mean M of the image. The calculation method of the global brightness mean M is to add the brightness value of each pixel point in the image, and then divide by the total number of pixels (640x480) of the image.

[0052] For the darker part of the image (when the value of the reflection component is less than or equal to the dynamic threshold T(M)), we have special enhancement parameters. One of the parameters will decrease with the increase of M, starting from 2.0, and decreasing by 0.015 for every certain M value; the other parameter will also adjust with the change of M, which is equal to 150 minus M. When M is less than 50, that is, in the condition of extremely low light, the first parameter can be reduced to 1.25, and the second parameter will be increased to 100, so that the difference between dark pixels can become more obvious, highlighting the details of the dark part.

[0053] For the brighter part of the image (when the value of the reflection component is greater than the dynamic threshold T(M)), there are also protection parameters. One parameter will increase with the increase of M, starting from 0.8, and increasing by 0.005 for every certain M value; the other parameter is equal to 30 plus 0.5 times M. Here, the dynamic threshold T(M) will gradually decrease as the light becomes dim, which can ensure that the enhancement effect of the dark part is more significant.

[0054] T(M) is negatively correlated with the global brightness mean M, and is a linear function wherein, is an initial threshold value, is an adjustment coefficient and k>0. For example, when M=50, T(M) can correspond to a certain intermediate value; when M<50, T(M) decreases with M to enhance dark processing.

[0055] In addition, a compensation constant is also introduced. Its role is to eliminate the problem of gray level mutation when the image is segmented, and to ensure that the transition from dark to bright is very smooth and there is no obvious trace.

[0056] 3. Texture adaptive weighted fusion strategy

[0057] First, the local texture standard deviation of each scale reflection component is calculated. Taking a 5x5 pixel neighborhood as an example, the sum of the squares of the difference between the reflection component data in the neighborhood and the neighborhood mean is calculated, and then divided by 25 to obtain. The neighborhood mean is the average value of the reflection component in the 5x5 neighborhood.

[0058] Then, the weight distribution is carried out, and the weight is determined according to the proportion of the square of the local texture standard deviation of each scale in the sum of the squares of all scale standard deviations. Then, the final reflection component is obtained by multiplying each scale reflection component by its corresponding weight and summing.

[0059] For different regions, the corresponding weight can exceed 70% in the texture-rich region (such as around the eyes, the local texture standard deviation is greater than 15); in the smooth region (such as the cheeks, the local texture standard deviation is less than 5), noise is suppressed by multi-scale averaging. In processing the reflection component of the low-light face image, there is also a dynamic enhancement strategy, which first calculates the global brightness mean, that is, the sum of the brightness values of all pixels is divided by the total number of pixels (640x480).

[0060] In order to make the image transition naturally at the segmented processing place, and not to appear gray level mutation, a compensation constant C is also introduced, which is calculated by a specific formula to ensure the overall smoothness of the image.

[0061] The texture adaptive weighted fusion strategy invention, in order to fuse reflection components of different scales, first calculates the local texture standard deviation σi of each scale reflection component in a 5x5 pixel neighborhood, which reflects the texture variation in the neighborhood. Then, the weight w i of each scale is calculated according to the standard deviation. The way of calculating the weight makes the scale with more texture change have a higher proportion in the fusion. Finally, the reflection components of the three scales are added according to their respective weights to obtain the fused result. In the texture-rich region, such as around the eyes, the weight of a certain scale can account for more than 70%, highlighting the details of these regions; while in the texture-smooth region such as the cheeks, multi-scale averaging is used to reduce noise interference

[0062] 2. Adaptive Feature Fusion Steps

[0063] 1. Dual-modal feature refinement extraction LBP feature enhancement: In the dual-modal feature refinement extraction, LBP feature enhancement is an important part. A circular template with a radius of 2 pixels and 8 neighborhoods is used to calculate the LBP value of the uniform pattern. A certain pixel in the image is taken as the center, denoted as (x c ,y c ), whose gray value is g c , the grayscale value of the surrounding 8 neighboring pixels is g p . Through a specific formula To calculate the LBP value, the s(x) function takes a value based on whether x is greater than or equal to 0. Then s(x) = 1, otherwise 0. In this way, each pixel is converted into an LBP value, and then a 59-dimensional uniform pattern histogram is generated. In order to make the features more standardized and comparable, they are also converted into unit vectors through L2 normalization. This series of operations can highlight the local texture features of the image, allowing subtle texture changes in the image to be clearly displayed, and provide strong feature support for subsequent image analysis and recognition tasks;

[0064] Pseudo-infrared HOG feature generation: A pre-trained U-Net model is used to convert low-light RGB images into pseudo-infrared images, enhancing the representation of thermal radiation differences. HOG feature extraction parameters: Cell size: 8x8 pixels; Block size: 16x16 pixels (50% overlap); Number of directions: 9 (0°-180°, 20° intervals); A 3780-dimensional feature vector is generated, capturing facial contours and edge information.

[0065] 2. Dynamic weight adjustment mechanism

[0066] Based on the global brightness mean M, the LBP feature weight is calculated by the Sigmoid function ;

[0067] When M<60 (low light), α>0.7, LBP texture features dominate; when When α is less than 0.4 (normal lighting), the HOG gradient feature weight is increased; the coefficient "0.1" determines the gradient of the weight α as it changes with the global brightness mean M. A smaller coefficient (such as 0.1) makes the function curve smoother, preventing the weight from fluctuating violently with slight changes in lighting; a larger coefficient (such as 0.5) makes the curve steeper and the weight switching more sensitive;

[0068] Calculation verification:

[0069] When M=60, ;

[0070] When M = 80, (theoretical value), the LBP feature weight a is less than 0.4 by the HOG gradient feature dominant feature normalization coefficient.

[0071] The fusion formula is: where F LBP and are the normalized LBP and pseudo-infrared HOG feature vectors, respectively. Specifically, for the extraction of LBP features to enhance the aspect, a specific circular template is used to enhance the LBP feature. The template radius is 2 pixels, and it has an 8-neighborhood. When calculating the uniform mode LBP value, the center pixel is taken as the reference, and its gray value is compared with the gray value of the neighborhood pixels. The center pixel is denoted as (x c ,y c ), the gray value is g c , and the gray value of the neighborhood pixels is g p . The formula is used for calculation, where s(x) takes the value 1 or 0 according to whether x is greater than or equal to 0. In this way, the LBP value corresponding to each pixel is calculated, and a 59-dimensional uniform mode histogram is generated. Finally, L2 normalization is performed, and the unit vector is converted, highlighting the local texture details of the image.

[0072] For the generation of pseudo-infrared HOG features, a pre-trained U-Net model is used to convert low-light RGB images into pseudo-infrared images, making the thermal radiation difference more obvious. Then, the HOG features are extracted, with a cell size of 8x8 pixels, a block size of 16x16 pixels and an overlap rate of 50%, and 9 directions from 0° to 180° with an interval of 20°. According to these settings, a 3780-dimensional feature vector is obtained to capture the contour and edge information of the face. The dynamic weight adjustment mechanism determines the LBP feature weight a through the Sigmoid function according to the global brightness mean M of the image. When M is less than 60, in low light, a is greater than 0.7, meaning that the LBP texture feature plays a major role in the fusion, because texture is important for recognition in low light. When M is greater than or equal to 80, in normal light, a is less than 0.4, and the HOG gradient feature weight increases at this time. The fusion formula is where F LBP and are the normalized LBP and pseudo-infrared HOG feature vectors, respectively, so that the two features can be adaptively fused according to the lighting conditions;

[0073] The latent structure of the U-Net model is an encoder-decoder structure, and the U-Net contains symmetric down-sampling (encoder) and up-sampling (decoder) paths, which are fused through jump connections to preserve multi-scale features. When converting RGB to pseudo-infrared, the following core design is used:

[0074] Encoder: consists of 4-5 down-sampling blocks, each block contains 2-3 convolution layers (e.g., 3x3 kernel), combined with Batch Normalization and ReLU activation function, gradually reduces the spatial resolution (e.g., from 640x480 to 80x60), extracts semantic features.

[0075] Decoder: corresponds to 4-5 up-sampling blocks, restores the resolution through transposed convolution or interpolation, combines with the bottom-layer details delivered by the skip connection, generates pseudo-infrared images.

[0076] Number of convolution kernels: the number of convolution kernels in the initial layer may be 64, and it is doubled (e.g., 64→128→256→512) after each down-sampling, and the corresponding layer in the decoder is halved again, which conforms to the typical configuration of U-Net;

[0077] Pseudo-infrared images aim to represent the thermal radiation difference of objects, while some pixel values (e.g., red channel) in RGB images have an indirect correlation with the surface temperature of objects (e.g., high-temperature objects are reddish). U-Net converts visible light images into simulated infrared thermal images by learning the mapping relationship between RGB pixel values and infrared radiation intensity.

[0078] III. Attention mechanism optimization steps

[0079] 1. Spatial attention module implementation

[0080] Feature compression and activation: global average pooling and global maximum pooling are performed on the fused feature to generate F avg and F max . For the fused feature (where H represents height, W represents width, and C represents the number of channels), the process of generating F avg and F max is as follows: generating F avg : through global average pooling operation, the fused feature F fusion is averaged in the spatial dimension (i.e., height H and width W dimension). Specifically, the pixel values at the corresponding position in each channel are added, and then divided by the total number of pixels in that channel (HxW), to obtain an average value vector in the channel dimension. These vectors are combined to form F avg , which compresses the fused feature and highlights the overall average feature information. Generating F max : using global maximum pooling operation, find the maximum pixel value at the corresponding position in each channel of the fused feature F fusion , combine these maximum values to form F max , which emphasizes the most significant feature information in the fused feature;

[0081] The input channel number of the convolutional layer is 2C, and the output channel number is 1, so the convolution kernel size is 7*7*2C, and the number is 1; the convolutional layer generates a single-channel spatial attention matrix W by weighting and summing 2C input channels s , the mathematical expression of which is

[0082] , wherein is a Sigmoid activation function, The weight matrix dimension of is 1*7*7*2C, which realizes the integration and transformation of cross-channel information.

[0083] When the 7*7 convolutional layer performs convolution operation on the 2C-dimensional feature, it learns the correlation between different channels through the weight matrix, for example, it strengthens the channel weight related to the face region (such as the eye, nose) and suppresses the interference of the background channel.

[0084] Convolution extracts spatial weights: a 7x7 convolutional layer (padding=3, activation function is Sigmoid) generates a spatial attention matrix : Highlighting key areas such as eyes (weight increased by 40%-60%) and nose. 7*7 convolutional layer: refers to the convolutional layer with a convolution kernel size of 7*7 pixels. Convolution operation is to slide the convolution kernel on the image (here is the feature map), and perform weighted summation on the elements at the corresponding positions, so as to extract image features. padding=3: indicates the number of pixels padded to the input feature map before convolution operation. Padding is to add additional pixels around the border of the feature map, here 3 pixels are added to the top, bottom, left and right of the feature map. The purpose is to keep the size of the feature map unchanged during convolution, to avoid excessive loss of edge information due to convolution operation. The activation function is Sigmoid: Sigmoid function is a commonly used activation function, the formula is , which can compress the input value to 0 to 1. Here, the Sigmoid function is used as the activation function to add non-linear characteristics to the output of the convolutional layer, so that the model can learn more complex patterns. s : that is, the spatial attention matrix, with a dimension of , H is the height of the feature map, and W is the width of the feature map. Each element value of the matrix represents the attention weight of the corresponding position in the spatial dimension, and the larger the value, the more attention the model pays to the position. : represents the Sigmoid activation function, which is used for non-linear transformation of the convolutional layer output to generate the element value of the spatial attention matrix. : represents the 7*7 convolution operation, which performs convolution operation on the concatenated feature to extract spatial features. Concat(F avg ,F max): is the F generated by global average pooling avg and F generated by global maximum pooling max Concatenate along the channel dimension (Concat is the abbreviation of Concatenate) to merge the two features into a new feature as the input of the subsequent convolution operation. avg : is the fusion feature F fusion The features generated after global average pooling highlight the average information of the fused features. max : is the fusion feature F fusion The features generated after global maximum pooling highlight the most significant information in the fused features.

[0085] 2. Channel Attention Module Optimization

[0086] Global information aggregation: perform global average pooling on Ffusion to obtain channel statistics ; z: channel statistics, dimension is , is the fusion feature F fusion The result obtained after global average pooling aggregates the global information on each channel. : The cth element in the channel statistic z is calculated by the formula Calculated, that is, the fusion feature F fusion The statistical value of each channel is obtained by summing the spatial dimensions (x from 1 to W, y from 1 to H) and dividing it by the total number of pixels H×W.

[0087] Nonlinear transformation: Generate channel weights through a two-layer fully connected network (dimensionality reduction ratio 16:1) ,in , is the ReLU activation function. It suppresses the background color channel (such as the blue channel) and enhances the skin texture channel (such as the green channel) by 20%-30%. c : Channel weight, dimension is , is the result obtained after processing by two layers of fully connected networks, used to indicate the importance of each channel; W2: the weight matrix of the second layer of fully connected networks, with a dimension of , used to further transform the features processed by the first layer of fully connected network. : ReLU activation function, the formula is , used to introduce nonlinearity in a two-layer fully connected network to enhance the expressive power of the model. W1: The weight matrix of the first layer of the fully connected network, with a dimension of , perform dimensionality reduction transformation on the channel statistics z. z: The channel statistics obtained by the previous global information aggregation are used as the input of the two-layer fully connected network.

[0088] The relationship between the original channel number and the number of intermediate layer nodes is: let the input channel statistics z e R C , C is the number of channels of the fused features, and the dimension reduction ratio of the two-layer fully connected network is 16:1, that is:

[0089] The output node number of the first layer fully connected layer (dimension reduction layer) is: u = C / 16, and the weight matrix W1 e R （C / 16）×C ;

[0090] The output node number of the second layer fully connected layer (dimension increase layer) is restored to C, and the weight matrix W2 e R C×（C / 16） ;

[0091] Mathematical expression: intermediate layer latitude = original channel number C / 16; for example, if the number of channels of the fused features C = 512, then the number of intermediate layer nodes is 512 / 16 = 32.

[0092] 3. Joint attention and residual fusion

[0093] Weight matrix fusion: , denotes element-wise multiplication;

[0094] Residual connection: wherein, is a shallow feature (such as a gradient feature of an original image) that maintains the same dimension through up-sampling and preserves edge details. In terms of attention mechanism optimization steps, this part focuses on optimizing the attention mechanism through a series of operations, mainly covering spatial attention module, channel attention module, and joint attention and residual fusion.

[0095] In terms of spatial attention module implementation, first, the fused features are processed through global average pooling and global maximum pooling to compress and activate the fused features, obtaining two different processed features.

[0096] Next, the two features are spliced and then passed through a 7x7 convolution layer (padding = 3, and the activation function is Sigmoid) to generate a spatial attention matrix. This matrix highlights the eye (weight increase by 40%-60%), nose, and other key areas of the face, so that the model pays more attention to the feature information of these important parts.

[0097] In terms of channel attention module optimization, first, the fused features are subjected to global average pooling to aggregate global information in the channel dimension, obtaining channel statistics.

[0098] Then the channel statistics are input into a two-layer fully connected network (dimension reduction ratio 16:1) for nonlinear transformation to generate channel weights. This operation will suppress the background color channels (such as the blue channel), while enhancing the weight of the skin texture channel (for example, the green channel) by about 20%-30%, achieving differential processing of different channel features.

[0099] In terms of joint attention and residual fusion, the spatial attention matrix and the channel attention matrix are multiplied element by element to fuse the information of the two and obtain a comprehensive weight matrix.

[0100] In the manner of residual connection, the fused features are multiplied by the weight matrix obtained above, and then the shallow features (such as the gradient features of the original image, which are made consistent in dimension with the fused features by upsampling) are added to retain the edge details of the image and optimize the final features.

[0101] IV. Technical details expansion

[0102] 1. Illumination evaluation preprocessing threshold setting: when M < 60 (corresponding to an illumination of about 5 lux), trigger the low-light enhancement process; scale parameter dynamic adjustment: ;

[0103] For example, when M = 30, decreases to 12, enhancing the ability to extract local details. In this formula: : is the scale parameter after dynamic adjustment. It will change according to the illumination of the image, and then affect the subsequent processing effect of the image, such as adjusting the ability to extract local details of the image. : is the initial value of the scale parameter, which is the original parameter setting before considering the dynamic adjustment of the illumination. e: natural constant, which is a mathematical constant, approximately equal to 2.71828, serving as the base of the exponential function in this formula. M: represents the global brightness mean of the image. It is obtained by calculating the brightness values of all pixels in the image and averaging them, and is used to judge the illumination of the image. When M < 60, the low-light enhancement process is triggered, and it is also a key variable in the scale parameter dynamic adjustment formula. The whole formula describes how the scale parameter is dynamically adjusted according to the global brightness mean M to adapt to the needs of image feature extraction under different illumination conditions.

[0104] 2. Noise suppression preprocessing

[0105] Non-local mean filter parameters: search window size: 21x21 pixel neighborhood block size: 5x5 pixels similarity weight: where, N p and N qFor the neighborhood blocks of pixels p and q, the L2 distance is calculated and normalized, effectively suppressing Gaussian noise (standard deviation reduced by 40%). Search window size (21x21 pixels): The search window defines the range in which to find similar pixel neighborhoods in the image. With a certain pixel as the center, a 21x21 pixel range around it is searched for similar pixel blocks. A larger search window means more pixel blocks are available for comparison, allowing more comprehensive capture of similar structural information in the image. For example, in a texture-rich image, a larger search window can find multiple similar texture pattern regions, providing more effective information for accurate noise suppression. However, a too large window will also increase the amount of calculation, so a balance between effect and calculation efficiency needs to be struck.

[0106] Neighborhood block size (5x5 pixels): The neighborhood block is the basic unit for measuring the similarity between pixels. For each pixel, a 5x5 pixel block centered on it is taken. Within the search window, the correlation between pixels is determined by comparing the similarity of 5x5 neighborhood blocks at different positions. For example, when judging the noise of a certain pixel, its 5x5 neighborhood block is compared with the 5x5 neighborhood blocks of other pixels in the search window. The size of this setting is also crucial, too small may not capture enough structural information to accurately judge similarity, too large may contain too much irrelevant information, affecting the accuracy of similarity judgment. p and N q : Respectively represent the 5x5 neighborhood blocks of pixels p and q. These two neighborhood blocks contain the pixel value information around the corresponding pixels, and the similarity between pixels p and q is determined by comparing the difference between them. : Here, the square of the L2 distance between neighborhood blocks Np and Nq is calculated. L2 distance (Euclidean distance) is a commonly used way to measure the distance between vectors, which in the image is to measure the distance between two neighborhood block pixel value vectors. Calculating the square of the L2 distance is to better reflect the difference between neighborhood blocks in the formula, and the square operation will amplify the difference, making the similarity judgment more sensitive. : In the denominator of the formula, it plays a role in adjusting the weight calculation. It can control the sensitivity of the weight to the difference between neighboring blocks. The larger the value, the less sensitive the weight is to the difference between neighboring blocks. Even if there are certain differences between neighboring blocks, they may be given a relatively high weight; the smaller the value, the more sensitive the weight is to the difference. The weight will be greatly reduced if there is a slight difference between neighboring blocks. Here it is set to , is a value that, through experimental and theoretical analysis, achieves a good balance between similarity judgment and noise suppression effectiveness in the current image noise suppression scenario. This non-local means filter parameter setting and calculation effectively suppresses Gaussian noise, reducing the standard deviation of the noise by approximately 40%, significantly improving image quality and providing cleaner, more accurate image data for subsequent image analysis and processing.

[0107] 3. Adaptive Histogram Equalization

[0108] Color space conversion: convert the image from RGB to YCbCr, processing only the brightness component Y;

[0109] Block size: 16x16 pixels, calculate the local cumulative distribution function (CDF): ; : Indicates the brightness value of the block after processing, which is the result of calculation. : Represents the local cumulative distribution function value calculated for the brightness component Y in a 16×16 pixel block. It reflects the distribution and accumulation of brightness values ​​in the block and is obtained through statistical analysis of the brightness data in the block. Y: is the brightness component of the original image after conversion to the YCbCr color space. Y max : Refers to the maximum value of the brightness component Y in the current block, that is, the pixel value with the highest brightness in the 16×16 pixel block. min : represents the minimum value of the luminance component Y within the current block, that is, the lowest brightness pixel value in the 16×16 pixel block. This formula adjusts the block's luminance value by combining the local cumulative distribution function and the maximum brightness value within the block, achieving adaptive histogram equalization.

[0110] Block effect removal: The block boundaries are smoothed by bilinear interpolation, and the interpolation kernel is a 3x3 Gaussian kernel.

[0111] 4. Liveness detection assistance

[0112] LTP feature: 105-dimensional histogram generated using ternary pattern of radius (r = 1, 8-neighborhood; optical flow field calculation: Farneback algorithm used to extract the mean and variance of the motion vectors of adjacent frames, generating 2-dimensional features; classifier: SVM (radial basis kernel function), training samples containing 100,000+ real faces and 50,000+ attack samples, confidence threshold set to 0.7, live detection accuracy reaching 97.2%. Specifically:

[0113] When processing images, the preprocessing link is crucial, as it lays the foundation for more accurate analysis and recognition in the subsequent steps.

[0114] First is the light evaluation preprocessing. By calculating the global brightness mean M of the image, the light condition can be judged. When M is less than 60, it means that the light intensity is weak, approximately corresponding to an illumination of 5 lux, at which point the low-light enhancement process is triggered. At the same time, the scale parameter is dynamically adjusted according to specific rules. For example, when M is 30, the scale parameter is reduced, from the original parameter to 12, which enhances the ability to extract local details of the image and helps better capture subtle features in low-light images.

[0115] Next is the noise suppression preprocessing. The non-local mean filtering method is used, with carefully set parameters. The search window size is determined to be 21x21 pixels, and the neighborhood block size is 5x5 pixels. In calculating the similarity weight between pixels, specific rules are followed. Through such settings, Gaussian noise can be effectively suppressed, reducing the standard deviation of noise by about 40%, making the image clearer and reducing the interference of noise on subsequent processing.

[0116] Then is the adaptive histogram equalization step. First, the image is converted from the common RGB color space to the YCbCr color space, because in this space, the luminance component Y is separated from the color components Cb and Cr, making it easier to process the luminance separately. Only the luminance component Y is processed, with a block size of 16x16 pixels, and the local cumulative distribution function is calculated. In this way, the luminance distribution characteristics of different regions of the image can be adjusted accordingly. To eliminate the block effect caused by block processing, a bilinear interpolation method is used, with a 3x3 Gaussian kernel as the interpolation kernel, to smooth the block boundaries, making the image transition natural and the visual effect better.

[0117] Finally, the living body detection auxiliary link. In this process, a variety of technical means are used. First, the three-value mode of the radius r = 1, 8 neighborhood is used to extract the LTP feature, and then a 105-dimensional histogram is generated, which can reflect some inherent characteristics of the face. Then the optical flow field is calculated by the Farneback algorithm, and the mean and variance of the motion vector of the adjacent frames are extracted, and a 2-dimensional feature is generated to capture the dynamic information of the face. These features are input into the support vector machine (using radial basis kernel function) for classification. The training samples include more than 100,000 real face samples and more than 50,000 attack samples, and the confidence threshold is set to 0.7. After such training and setting, the accuracy of the living body detection can reach 97.2%, which can effectively judge whether the face is a living body and provide reliable protection for subsequent face recognition and other applications.

[0118] The face image enhancement and recognition method in low light environment provided by the application realizes the whole process optimization from image preprocessing to identity judgment through multi-technology fusion, which is as follows:

[0119] In the preprocessing stage, the global brightness mean M of the image is calculated, and when M < 60, the enhancement process is triggered. A non-local mean filtering algorithm with a 21x21 pixel search window and a 5x5 pixel neighborhood block is used to suppress Gaussian noise, so that the noise standard deviation is reduced by 40%. After converting the image to YCbCr space, the local histogram equalization of the 16x16 pixel block of the brightness component is performed, and the block effect is eliminated by 3x3 Gaussian kernel bilinear interpolation to improve the local contrast.

[0120] The image enhancement core link adopts a three-scale Retinex technology: with sigma = 15 (31x31 kernel) to extract high-frequency details such as eyelashes, sigma = 80 (161x161 kernel) to capture facial organ contours, and sigma = 250 (501x501 kernel) to analyze global illumination. After decomposing the illumination and reflection components in the logarithmic domain, an adaptive piecewise logarithmic transformation function is used to enhance the reflection component, where the dark part parameter gamma1 decreases linearly from 2.0 with a step size of 0.015, and beta1 = 150-M; the bright part parameter gamma2 increases from 0.8 with a step size of 0.005, and beta2 = 30 + 0.5M, the dynamic threshold T(M) decreases linearly as the light darkens, and the weight of each scale reflection component is dynamically adjusted according to the texture standard deviation of the 5x5 neighborhood, and the weight of the texture-rich area is more than 70%.

[0121] In the feature fusion stage, 59-dimensional LBP texture features and 3780-dimensional pseudo-infrared HOG features (extracted from pseudo-infrared images generated by U-Net) are extracted, and the fusion weight is dynamically adjusted using the Sigmoid function alpha = 1 / (1+e^(-0.1x(M-80))) : when M < 60, alpha > 0.7, LBP dominates; when M >= 80, alpha < 0.4, HOG dominates.

[0122] In the attention optimization module, the spatial attention is processed by 7x7 convolution (padding=3) on the features after the global average pooling and maximum pooling are spliced to generate a spatial weight matrix focusing on key areas such as the eye (weight increase of 40%-60%); the channel attention is calculated by a full connection layer of C→C / 16→C to enhance the skin texture channel by 20%-30% and suppress the background channel. After element-wise multiplication, the residual connection of the shallow layer features is connected, and the discriminative feature vector is output.

[0123] In the post-processing stage, the horizontal and vertical gradient of the enhanced image is calculated to evaluate the definition, and the parameters are iteratively adjusted when the definition is lower than the threshold; 105-dimensional LTP features, 2-dimensional optical flow features and pseudo-infrared HOG features are spliced into a 3887-dimensional vector, which is input into the SVM (radial basis kernel) for live body detection, and the accuracy is 97.2%. Finally, the optimized features are input into the pre-trained model to complete the recognition through cosine similarity matching. Experiments show that the recognition accuracy of this method is 85.3% under ≤5 lux environment, which is improved by 102.6% compared with the traditional method, the false recognition rate is reduced to 2.3%, the definition Q value is improved by 125.6%, and the processing delay is optimized to 85ms / frame, which meets the real-time security requirements.

[0124] Test data and beneficial representation

[0125] Data interpretation: The present application improves the feature recognition degree by dynamic segmentation logarithmic transformation (T(M) adjusts adaptively with light) and texture weighted fusion (w i The recognition accuracy is doubled. The cross-layer residual attention module has the dual effects of spatial attention (eye weight increase of 50%) and channel attention (skin texture channel enhancement of 25%), which greatly reduces the false recognition rate. The combination of non-local mean filtering and adaptive histogram equalization improves the image definition Q value by 125.6%, and the processing delay is optimized to 85ms, which meets the real-time monitoring requirements. The live body detection module effectively resists photo and video attacks through multi-modal feature fusion (LTP+optical flow+pseudo-infrared HOG), and the recognition rate is improved to 97.2%. The overall scheme has achieved a comprehensive breakthrough in face image enhancement and recognition performance in low light environment, and is suitable for smart security, mobile payment and other scenarios with extremely high environmental adaptability requirements.

[0126] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can make equivalent substitutions or changes within the technical scope disclosed by the present application according to the technical solutions and inventive concepts of the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for enhancing and recognizing facial images in a low-light environment, characterized in that: The following steps are involved: Multi-scale Retinex enhancement: The input low-light face image is subjected to three-scale Gaussian filtering, with the scale parameters set to 15, 80, and 250, respectively. This filtering separates the illumination and reflection components at different scales. The reflection component is nonlinearly enhanced using an adaptive piecewise logarithmic transformation function. This function dynamically adjusts the threshold and compensation parameters based on the global brightness mean of the image, enhancing dark details while suppressing noise amplification. Adaptive feature fusion steps: Extract the local binary pattern (LBP) features and the histogram of oriented gradients (HOG) features of the enhanced image, evaluate the lighting conditions by calculating the global brightness mean of the image, and dynamically adjust the fusion weights of the two types of features: automatically increase the weight of the LBP texture features in weak lighting to highlight subtle facial textures; increase the weight of the HOG gradient features in strong lighting to enhance contour information; and generate a feature vector of the information through a dynamic fusion strategy; Attention mechanism optimization steps: Construct a residual attention module, extract spatial features through convolution operations and generate a spatial attention weight matrix to focus on the face area; at the same time, use global average pooling and fully connected layers to calculate the channel attention weight matrix to suppress the interference of background channels; after multiplying the spatial and channel attention weight matrices element by element, perform weighted optimization on the fused features to improve the feature expression ability of the region and output a discriminative feature vector.

2. The method for enhancing and recognizing facial images in a low-light environment according to claim 1, wherein: Also includes: Lighting assessment preprocessing step: Calculate the global brightness mean of the input image. When the mean is lower than the set threshold, the enhancement process is triggered. The scale parameters of the multi-scale Retinex enhancement are dynamically adjusted based on the brightness mean to achieve image enhancement requirements under different light intensities.

3. The method for facial image enhancement and recognition in a low-light environment according to claim 1, wherein: Also includes: Noise suppression preprocessing step: The input image is denoised using the non-local mean filtering algorithm. By calculating the similarity weight between the current pixel and the pixels in the search window, the image edge details are retained while suppressing noise to generate a denoised image.

4. The method for enhancing and recognizing facial images in a low-light environment according to claim 1, wherein: In the multi-scale Retinex enhancement step, when weighted fusion is performed on reflection components of different scales, the weights are dynamically adjusted according to the local texture features of the image, and the scale components corresponding to the texture area are given higher weights.

5. The method for enhancing and recognizing facial images in a low-light environment according to claim 1, wherein: Also includes: Adaptive histogram equalization step: convert the enhanced image to YCbCr color space, perform block-wise local histogram equalization on the luminance component to improve local contrast; The blocking effect at the sub-region boundary is eliminated by bilinear interpolation.

6. The method for enhancing and recognizing facial images in a low-light environment according to claim 1, wherein: In the attention mechanism optimization step, the spatial attention module extracts spatial features through convolution operations, and the channel attention module calculates channel weights through global average pooling and fully connected layers, and finally generates an attention weight matrix through element-by-element multiplication.

7. The method for facial image enhancement and recognition in a low-light environment according to claim 1, wherein: Also includes: Face recognition and matching steps: The optimized feature vector is input into the pre-trained face recognition model to extract the face feature vector and calculate the cosine similarity with the registered vector in the database. The success of face recognition is determined based on the similarity value.

8. The method for facial image enhancement and recognition in a low-light environment according to claim 1, wherein: Also includes: Quality assessment post-processing steps: Calculate the clarity index of the enhanced image and evaluate the image quality based on the gradient changes in the horizontal and vertical directions; When the clarity is lower than the threshold, the enhancement parameters are iteratively adjusted to meet the quality requirements.

9. The method for facial image enhancement and recognition in a low-light environment according to claim 1, wherein: Also includes: Auxiliary steps for liveness detection: Extract the local ternary pattern LTP texture features and optical flow field micro-motion features of the enhanced image, and input them into the liveness detection model for authenticity judgment to resist non-liveness attacks in photos and videos.

10. The method for enhancing and recognizing facial images in a low-light environment according to claim 1, wherein: The piecewise logarithmic transformation function used in the multi-scale Retinex enhancement step dynamically adjusts the enhancement parameters according to the image brightness to achieve processing of different lighting areas.

Images