Feature decoupling and fusion-based low-illumination face recognition method and system

Abstract

The invention discloses a feature decoupling and fusion low-illumination face recognition method and system. The method comprises the following main steps: S1, constructing a feature decoupling and fusion face recognition method according to a relationship between image illumination and facial features; s2, training is carried out by adopting non-paired face data, and a lighting triple is constructed; s3, introducing a decoupling module, decomposing the image into illuminance-related features and face-related features, and eliminating the influence of an illuminance factor on the performance of a face recognition system through illumination triple loss; and S4, introducing a feature fusion module, extracting residual face features in the illumination related features, and fusing the residual face features with the initial face related features. An illumination-face related loss is introduced. And S5, obtaining a result of the model on the data set, and calculating an Accuracy index. The method has higher generalization ability in a low-light scene. Compared with similar low-illumination face recognition methods, the method has higher recognition precision.

Description

Technical Field

[0001] The present invention relates to computer vision, face recognition and other related fields, and in particular to a method and system for enhancing facial features based on eliminating lighting features. Background Art

[0002] Face recognition is one of the important technologies in the field of computer vision. With the support of deep learning technology, face recognition has achieved great success. As a way of identity authentication, face recognition technology is widely used in various fields of life. Compared with other identity authentication methods (such as fingerprint recognition, iris recognition and voice recognition, etc.), face recognition has the advantages of non-contact, convenience, compatibility and scalability. With the popularization of cutting-edge technologies such as deep learning, artificial intelligence, and big data, the field of identity recognition is ushering in a higher level of development. In daily life, face recognition technology is everywhere and plays a vital role. However, the traditional face recognition system based on face scanning machine is subject to many constraints, including: lighting changes, face occlusion and head posture and other difficult to control factors, among which the lighting change factor is more difficult to control. In some night scenes or dimly lit indoors, the features of the collected face images are not obvious, and they cannot be effectively matched with the faces in the database. Therefore, the traditional face recognition method has many limitations and inconveniences, especially when used in some public places or transportation. It not only reduces the traffic efficiency in public places, but also requires a lot of equipment costs for supplementary lighting.

[0003] As a means of identity authentication, face recognition can be seen everywhere in our daily lives. Under ideal conditions, that is, the person being collected is facing the collection device, the collector's head is in a natural state, and the collection environment is well-lit, the face recognition system can work perfectly. However, when there is a certain collection deviation, the recognition performance of the face recognition system will decline, and even recognition may fail. Camera-based face recognition can improve the efficiency of identity matching, but because the photos are taken in an unconstrained environment, the image quality is greatly affected by the uncontrollable factor of lighting. Therefore, how to improve the performance of face recognition systems in low-light environments has become a hot research topic in the field of face recognition technology in recent years.

[0004] In order to improve the performance of face recognition systems in low-light environments, near-infrared (NIR) images can be used. Unlike visible light (VIS) images, NIR images can emit NIR rays to ignore most natural lighting. However, near-infrared shooting not only requires additional equipment to capture near-infrared images, but also requires collecting faces at the best position, so the near-infrared method has certain limitations. With the emergence of deep learning technology and its wide application in the field of face recognition, face recognition technology has made substantial breakthroughs. At present, the recognition accuracy of some general face recognition models on high-quality face datasets has reached more than 95%, or even around 99%, which also shows that face recognition technology is quite mature under ideal conditions. However, these models still do not fully solve the performance problems in constrained scenarios. Their performance in low-light scenes is not optimal. Existing work points out that the performance of face recognition systems on low-light images will drop by more than 10%, thereby affecting the performance of face feature extraction. In the real world, solving the impact of illumination changes on the performance of face recognition systems is still a challenging problem.

[0005] Therefore, our goal is to develop a method for extracting robust facial features in low-light environments to improve the performance of face recognition systems in low-light conditions. Existing solutions for enhancing the performance of face recognition systems in low-light scenarios mainly include three approaches: 1) Use traditional illumination enhancement techniques (such as histogram equalization, adaptive histogram equalization, gamma correction, and deep learning-based methods) to enhance low-light face images as a preprocessing step, or use deep learning-based enhancement techniques to improve image brightness and contrast. These enhanced images are then passed to the face recognition system for subsequent recognition tasks. However, these methods may cause certain areas in the image to be too bright and noise amplified, potentially degrading the recognition performance of the face recognition system. 2) Use Retinex theory to normalize illumination changes, decompose face images into illumination and reflection components, and estimate these components to simulate how the human visual system processes images, thereby maintaining the visual quality of the enhanced image under different illumination conditions. However, this method mainly processes dual-component features in the image, resulting in the loss of facial texture feature information when searching for dual-component features. 3) The method based on NIR equipment can effectively image under low light conditions by capturing facial images in the near-infrared spectrum. However, NIR is limited by skin reflectivity, has limited anti-interference ability, and requires additional equipment costs.

[0006] To solve the above problems, the goal of this paper is to identify lighting feature information independent of facial features. In the Retinex theory, images are decomposed into reflection and illumination components to explain how the human eye perceives color and maintains a stable perception of object color under different lighting conditions. Inspired by the Retinex theory, we decouple face images into face-related features and illumination-related features, and use constrained supervised illumination features to mitigate the impact of illumination. The advantage of this design is that the model can process images directly on the captured face image device, thereby minimizing data transmission and storage. In addition, the use of feature enhancement eliminates the impact of illumination factors on the face recognition system, avoiding the noise or overexposure problems caused by pixel-by-pixel enhancement of low-light images. Overall, this method is of great significance and value for the application of face recognition systems in unconstrained environments and low-lighting scenarios, as well as promoting the high development of face recognition technology.

[0007] After searching, the application publication number CN106446872A is a method for video face detection and recognition under low illumination. Aiming at the specific problem of face recognition under low illumination, this system designs a video system that meets the face recognition function under low illumination. The algorithm of this system includes three steps: 1. Preprocessing of low illumination face video images based on fuzzy theory, 2. Training of detectors in the face detection algorithm based on Adaboost, 3. Modeling video face recognition based on feature fusion. The final face recognition result under low illumination is obtained. The method proposed in the present invention can actually be embedded in FPGA for implementation and used in a camera or video camera monitoring system with real-time face recognition function under low illumination. The method has good stability, high recognition rate, and fast computing speed. It can effectively detect and identify facial features under low illumination conditions. It can be applied to systems such as night vision monitoring and identity authentication under low illumination, and has good practicality.

[0008] Patent CN106446872A proposes a low-light video face recognition method based on fuzzy theory and Adaboost detection algorithm, which is innovative and practical. However, this method may still have some defects or shortcomings in practical applications, including the following:

[0009] 1. Limitations of low-light preprocessing methods

[0010] The patent adopts a low-light preprocessing method based on fuzzy theory. Although it can enhance the visibility of low-light images to a certain extent, the processing method of fuzzy theory may be limited in complex lighting environments. For example, under extremely low-light and strong noise conditions, fuzzy theory may not be able to effectively remove noise or enhance key features, thus affecting the accuracy of detection and recognition.

[0011] Improvement points: The present invention can adopt more advanced low-light enhancement algorithms, such as image enhancement models based on deep learning (e.g., Zero-DCE). These methods can improve image quality and retain more details by learning feature conversion from low illumination to high brightness.

[0012] 2. Adaptability of Adaboost detection algorithm

[0013] This patent uses a face detection algorithm based on Adaboost. Although Adaboost performs well under certain conditions, its detection performance may drop significantly under extremely low illumination or non-ideal illumination, and it is not robust enough for different facial postures and expression changes. In addition, the Adaboost algorithm has poor detection effect on small faces in complex backgrounds.

[0014] Improvements: The present invention can adopt more advanced deep learning detection algorithms, such as MTCNN or RetinaFace, which perform better in low-light conditions, can more effectively locate faces of different angles and sizes, and improve the robustness and accuracy of detection.

[0015] 3. Limitations of feature fusion methods

[0016] The recognition part of the patent uses a traditional method based on feature fusion, which may not be accurate enough when recognizing low-light faces in complex environments. In addition, the traditional feature fusion method may lose detailed information in low light during feature extraction and matching, affecting the recognition effect.

[0017] Improvement points: The present invention can introduce a feature decoupling and fusion network based on deep learning, such as DeFFace. This type of network can more effectively extract and fuse facial features under low-light conditions, and decouple lighting factors, further improving recognition accuracy and robustness.

[0018] 4. Real-time performance and hardware implementation limitations

[0019] The patent proposes that the method can be embedded in FPGA, but does not elaborate on the specific hardware implementation method and efficiency. FPGA implementation of complex algorithms may lead to increased latency and power consumption, especially for embedded devices, which may have resource constraints.

[0020] Improvements: The present invention can use lightweight deep learning models, such as MobileFaceNet, and combine them with FPGA optimizations, such as model pruning and quantization, to further improve the real-time performance and computing efficiency of the system, ensuring efficient face recognition in embedded environments. Summary of the invention

[0021] The present invention aims to solve the above problems of the prior art. A low-light face recognition method and system based on feature decoupling and fusion is proposed. The technical solution of the present invention is as follows:

[0022] A low-light face recognition method based on feature decoupling and fusion, comprising the following steps:

[0023] S1. Construct a triplet of face dataset, that is, take low-light face image as anchor, take low-light face image as positive sample, and normal brightness image as negative sample; shorten the feature space distance between anchor and positive, and increase the feature space distance between anchor and negative;

[0024] S2. In the feature extraction stage, the image is processed by the feature extraction module to generate a 128-dimensional feature vector; the feature extraction module is used to extract the basic features of the face;

[0025] S3, the feature vector then enters the decoupling module, which includes convolution Conv, batch normalization BN and ReLU activation function; the decoupling module is used to further optimize the feature representation and extract more representative facial features, thereby enhancing the model's ability to recognize subtle facial features;

[0026] S4, the decoupled facial features are fed into the fusion module; the fusion module integrates the signals from different features through average pooling, linear transformation and sigmoid activation steps; this process generates the final facial feature representation;

[0027] S5. The final facial features are input into the facial recognition module, which is responsible for classification and recognition. Different faces are accurately classified by inputting the extracted and fused features into the recognition module, thereby achieving facial recognition.

[0028] Furthermore, constructing the face data set triplet in step S1 specifically includes:

[0029] A1. Detect the key points of the face image through the face detection alignment network MTCNN. The face detection alignment network MTCNN refers to a multi-task convolutional neural network used to detect the face and its key points including eyes, nose, and mouth. MTCNN simultaneously performs face detection and key point positioning through a cascade structure, thereby achieving efficient and accurate face alignment. This network can effectively identify and locate facial features when processing images; and align the face image and crop it into an input image I of size 128×128;

[0030] A2. Set up a low-light face pool and a normal-light face pool, randomly select a face image from the low-light face pool as an anchor, and then randomly select another low-light sample as a positive; randomly select one from the normal-light face pool as a negative; these three images will form a facial triplet based on illumination.

[0031] Furthermore, the feature extraction method in step S2 specifically includes:

[0032] B1. Input the face image into the feature extractor Mobilefacenets, pass through an initial convolution layer to extract low-level features, and then transfer it to the depthwise separable convolution layer and the bottleneck block Bottleneck Blocks, which consists of two depthwise separable convolution layers connected by a linear transformation in the middle;

[0033] B2. A global average pooling layer is used to reduce the spatial dimensions of each feature map, i.e., width W and height H, to 1×1, and the average value of each channel is generated to obtain a feature vector of fixed length.

[0034] Furthermore, the decoupling module in step S3 specifically includes:

[0035] Given an input face image set Data = {low / dark, normal}, low / dark is a low-light face image set, normal is a normal-brightness face image set, and each image contains their identity ID; the purpose of the decoupling module is to separate independent illumination features and identity features, and there is no need to pay attention to the illumination label in this module; in the training stage, the decoupling module is trained by a given face image set; the goal of D-Net is to decouple the input image features into two components, and to separate the image illumination information from the facial information. This process is expressed as:

[0036]

[0037] Among them, I is the input face image, F is the decoupled 512-dimensional feature, when i=1, F is the decoupled 512-dimensional face feature, when i=2, F is the decoupled illumination feature, and D-Net represents the decoupling module; the input feature map passes through the continuous convolution layers in the two sub-modules F1 and F2 respectively to obtain rich abstract features, and then through the ReLU activation function, nonlinear processing is added to enhance the model's learning ability for facial features and illumination features, and finally a global average pooling layer is used to compress the features to unify the dimensions of the output features of the two sub-modules.

[0038] Furthermore, the feature fusion module in step S4 specifically includes the following steps:

[0039] A1: Feature extraction: Extract residual facial features from illumination features, which can reflect the shape and texture of the face; use convolutional neural network (CNN) to process images under different illumination conditions to ensure that the extracted features have high discrimination;

[0040] A2: Global pooling: Global pooling is performed on the extracted residual facial features to generate a feature vector of fixed size;

[0041] A3: Importance weight learning: The globally pooled features are input into the fully connected layer, and the importance weights of each channel are learned through training. During this process, the model will continuously adjust the weights through the back-propagation algorithm to strengthen the expression of important features while suppressing noise and redundant information.

[0042] A4: Feature separation and calibration: Separate the facial features from the illumination features based on the learned weights; then, use element-wise multiplication to fuse the calibrated illumination features with the initial facial features;

[0043] A5: Optimization of fusion features: Through multiple iterations and optimization, the quality of fusion features is further improved; regularization technology is introduced to prevent overfitting and ensure the generalization ability of the model;

[0044] This process is described as:

[0045]

[0046] Among them, X represents the input illumination feature map, Y represents the residual facial features in the illumination feature map, σ is a Sigmoid function, which is usually used to limit the output between 0 and 1, and W1 and W2 represent two linear transformations, W1 is used to reduce the number of channels, and W2 is used to restore the number of channels.

[0047] Furthermore, in step S5, the processed facial features are transmitted to the face matching module for subsequent recognition and matching, which specifically includes:

[0048] The face matching module first receives the processed high-dimensional feature vector output from F-Net and compares it with the stored features in the face database; the face matching module uses a variety of similarity measurement methods to perform matching evaluation; during the matching process, the similarity score is calculated for the input feature vector and the feature vector of each sample in the database; then, through the pre-set similarity threshold, when the matching score reaches or exceeds the threshold, the current input face feature successfully matches a face sample in the database and returns the corresponding identity information; in the case of matching failure, a "no match result" prompt is returned, and the unrecognized features are recorded for subsequent analysis and processing; at the same time, it is dynamically updated based on the learned sample features to continuously improve the feature vectors in the database.

[0049] A low-light face recognition system based on feature decoupling and fusion, comprising:

[0050] Face dataset triplet construction module: used to construct face dataset triplet, that is, low-light face image as anchor, low-light face image as positive sample, normal brightness image as negative sample; shorten the feature space distance between anchor and positive, and increase the feature space distance between anchor and negative;

[0051] Feature extraction module: In the feature extraction stage, the image is processed by the feature extraction module to generate a 128-dimensional feature vector; the feature extraction module is used to extract the basic features of the face;

[0052] Decoupling module: used for feature vector input decoupling module, which includes convolution Conv, batch normalization BN and ReLU activation function; the decoupling module is used to further optimize feature representation and extract more representative facial features, thereby enhancing the model's ability to recognize subtle facial features;

[0053] Fusion module: used to feed the decoupled facial features into the fusion module; the fusion module integrates the signals from different features through average pooling AvgPool, linear transformation and Sigmoid activation steps; this process generates the final facial feature representation;

[0054] Facial recognition module: The final facial features are input into the facial recognition module, which is responsible for classification and recognition; different faces are accurately classified by inputting the extracted and fused features into the recognition module, thereby achieving facial recognition.

[0055] The advantages and beneficial effects of the present invention are as follows:

[0056] 1. For problems related to face recognition under low-light conditions, existing methods mostly focus on image enhancement or feature extraction, but often fail to effectively reduce the interference of lighting changes on recognition results. Especially in the feature fusion stage, existing methods do not fully consider the correlation between lighting features and facial features. The present invention introduces lighting-face correlation loss during feature fusion, and effectively reduces the correlation between facial features and lighting features in the feature space by minimizing the cosine similarity between the two, thereby improving the accuracy and robustness of face recognition under low-light conditions.

[0057] 2. A low-light face recognition method based on feature fusion is proposed, which regards the fusion of illumination features and facial features as the key step to optimize feature representation. Through the improved feature selection algorithm, the features are classified and optimized, which further improves the purity and recognition effect of the features. This method transforms the feature fusion problem into an optimization problem of feature selection, ensuring that the selected features can maximize the recognition performance, thereby making full use of the model's learning ability and storage resources.

[0058] A low-light face decoupling model is proposed to reduce the impact of lighting factors on the face recognition system through feature enhancement, so as to improve the recognition performance of the face recognition system in low-light environments.

[0059] A D-Net is introduced to decouple the image into face-related features and lighting-related features. In addition, we introduce a new light triplet loss method based on the traditional triplet loss method, which uses unpaired ID training to control the lighting-related features in the decoupled module. F-Net is also introduced to further obtain pure face-related features, and Lighting-Facial Correlation loss is introduced to reduce the correlation between the two features.

[0060] The effectiveness of the proposed method is verified by comparing the existing low-light face recognition methods and several general face recognition methods on 4 real-light datasets and 3 simulated-light datasets. The experimental results show that the proposed method achieves convincing results, proving the effectiveness and superiority of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a feature fusion model diagram constructed by a preferred embodiment of the present invention;

[0062] Figure 2 It is a pseudo code of the specific method of screening lighting triples of the present invention;

[0063] Figure 3 It is the D-Net visualization view of the present invention.

[0064] Figure 4A schematic diagram showing the face recognition method in a low-light environment of the present invention. DETAILED DESCRIPTION

[0065] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0066] The technical solution of the present invention to solve the above technical problems is:

[0067] like Figure 1 As shown, the present invention provides a face recognition method in a low-light environment, comprising the following steps:

[0068] S1, builds an illumination-facial feature decoupling model based on illumination conditions and facial features. This model collects facial images under different illumination conditions, generates illumination feature maps, and then defines the relationship between illumination features and facial features. This step involves extracting useful feature information from data collected under various illumination conditions, laying the foundation for subsequent analysis.

[0069] S2, construct and optimize the illumination triplet. By analyzing the face images under different illumination conditions, the corresponding feature information is extracted. Specifically, a triplet (X, Y, Z) is constructed, where X represents the input illumination feature map, Y represents the residual facial features extracted from the illumination features, and Z is the optimized feature representation. This step aims to improve the effect of subsequent feature fusion by optimizing the features.

[0070] S3, feature fusion, fuses the residual facial features extracted from the illumination features with the initial facial features to further extract pure facial-related features. In this process, global pooling and fully connected layers are used to learn the importance weights of each channel, separate the facial features from the illumination features, and recalibrate them through element-wise multiplication to obtain a facial feature representation with stronger anti-interference ability. The goal of this step is to improve the purity and recognition accuracy of facial features.

[0071] S4, introduces Lighting-Facial Correlation Loss to reduce the correlation of decoupled features in the feature space. By minimizing the cosine similarity between facial features and lighting features, the two are de-correlated in the feature space, further reducing the interference of lighting features on facial recognition, and ultimately improving the recognition effect under low-light conditions.

[0072] S5, based on the final result, the system makes a decision on whether to cache or not cache each feature selection, ensuring that users can quickly obtain high-quality face recognition results when requested, regardless of changes in lighting.

[0073] S1, based on the brightness information of the image, the image is initially divided into a normal lighting face image pool and a low lighting face image pool. Randomly select images from the image pool to form lighting triplets. Specifically, the low lighting sample is used as the Anchor, the same low lighting sample is used as the Positive, and the normal lighting sample is used as the Negative. We aim to narrow the distance between the Anchor and the Positive, and widen the distance from the Negative, so as to achieve clustering between similar low lighting samples and distinction between different lighting samples. This process can be described as:

[0074] T i =Anchors m +Positive n +Negative k

[0075] Where i represents the i-th triplet, m and n represent any image in the low-lighting image pool, m is not equal to n, and k represents an image in the normal-lighting pool. Figure 2 The specific pseudo code for filtering lighting triples is given: This process can be expressed as:

[0076] T i ={Anchors m , Positives n , Negative k},

[0077] S2, feature decoupling, after the initial extraction of facial image features, it is input into the feature decoupling module (D-Net). The D-Net module is responsible for decoupling the features of the input image into "face-related features" and "lighting-related features". D-Net consists of two parallel sub-modules: L module and F module, which extract lighting and facial features respectively. Through the processing of D-Net, it is ensured that facial features are effectively extracted under low-light conditions without being disturbed by lighting. This process can be expressed as:

[0078]

[0079] Where I is the input face image, and F is the decoupled 512-dimensional feature. When i=1, F is the decoupled 512-dimensional face-related feature; when i=2, F is the decoupled 512-dimensional illumination-related feature. D-Net stands for the decoupled module. D-Net is visualized as follows: Figure 3 As shown:

[0080] S3, construction and optimization of illumination triples.

[0081] A1: Use the decoupled illumination features to construct the illumination triplet loss (Illumination TripletLoss). The goal of this loss is to effectively suppress the influence of illumination by controlling the distance between the anchor point and the positive sample in the triplet to be close, and the distance between the anchor point and the negative sample to be increased. Specifically, a low-light sample is used as an anchor point, another low-light sample is used as a positive sample, and a normal-light sample is used as a negative sample to control the influence of illumination features on the model. This process can be expressed as:

[0082]

[0083] Among them, A represents the anchor point, Am represents any low-light face sample in the low-light image pool; P is the positive sample, Pk represents any low-light face sample in the low-light image pool, and m≠k; N is the negative sample, η represents the sample distance factor, which is a fixed value of 0.5, and Nn represents any normal brightness sample in the normal lighting image pool.

[0084] A2: Optimization goal and process,By minimizing the illumination triplet loss function, the optimization goal of the present invention is to make the decoupled anchor illumination features more similar to the illumination features of similar low-light positive samples, while keeping the distance from the illumination features of normal-light negative samples as much as possible. The specific optimization process is as follows:

[0085] 1) Calculate the distance between the anchor point and the positive sample, and constrain the distance through the decoupling module to minimize it so as to achieve clustering of samples with similar lighting conditions.

[0086] 2) Calculate the distance between the anchor point and the negative sample, and control the expansion of this distance through the decoupling module to ensure that the normal light samples are far away from the low light samples in the feature space.

[0087] 3) Using the boundary value η in the loss function, we ensure that the difference between the positive and negative sample distances is always greater than the threshold, so as to effectively distinguish different lighting conditions in the feature space.

[0088] S4, feature fusion, fuses the residual facial features extracted from the illumination features with the initial facial features to further extract pure facial-related features. The importance weights of each channel are learned through global pooling and fully connected layers, the facial features in the illumination features are separated, and recalibrated through element-level multiplication to obtain a facial feature representation with stronger anti-interference ability. This step aims to improve the purity and recognition accuracy of facial features.

[0089] A1: Feature extraction: Extract residual facial features from illumination features, which can reflect the shape and texture of the face. Use convolutional neural network (CNN) to process images under different illumination conditions to ensure that the extracted features have high discrimination.

[0090] A2: Global pooling: Global pooling is performed on the extracted residual facial features to generate a fixed-size feature vector. This process effectively reduces the dimension of the feature while retaining important position information, making the feature more compact and easier to process.

[0091] A3: Importance weight learning: The globally pooled features are input into the fully connected layer, and the importance weights of each channel are learned through training. During this process, the model will continuously adjust the weights through the back propagation algorithm to strengthen the expression of important features while suppressing noise and redundant information.

[0092] A4: Feature separation and calibration: Based on the learned weights, the facial features are separated from the illumination features. Next, the calibrated illumination features are fused with the initial facial features using element-wise multiplication. This step can effectively enhance the anti-interference ability of the facial features and ensure that the fused features remain stable in the face of various illumination changes.

[0093] A5: Optimization of fusion features: Through multiple iterations and optimization, the quality of fusion features can be further improved. You can consider introducing regularization technology to prevent overfitting and ensure the generalization ability of the model.

[0094] This process is described as:

[0095]

[0096] Among them, X represents the input illumination feature map, Y represents the residual facial features in the illumination feature map, and W1 and W2 represent two linear transformations. W1 is used to reduce the number of channels, while W2 is used to restore the number of channels.

[0097] S5, correlation loss constraint, after feature fusion, introduces lighting-facial correlation loss to reduce the correlation of decoupled features in the feature space. By minimizing the cosine similarity between facial features and lighting features, the decorrelation of the two in the feature space is achieved, further reducing the interference of lighting features on facial recognition, and ultimately improving the recognition effect under low-light conditions. This process is expressed as:

[0098]

[0099] Preferably, the specific method of introducing the illumination-face correlation loss in step S5 includes:

[0100] After feature fusion, the cosine similarity between facial features and illumination features is first calculated to quantify the correlation between the two in feature space. Cosine similarity is a metric between -1 and 1, and the closer the value is to 0, the lower the correlation between facial features and illumination features. By minimizing this similarity, our goal is to effectively reduce the interference of illumination changes on face recognition. During model training, illumination-face correlation loss will be introduced into the total loss function together with other loss functions (such as classification loss and reconstruction loss). This combined optimization strategy ensures that the model reduces the mutual influence between facial features and illumination features while improving recognition accuracy. Using the back-propagation algorithm, the model will automatically adjust the network parameters to gradually achieve the decorrelation of facial features and illumination features in the feature space. In addition, in order to evaluate the effect of illumination-face correlation loss, the change of loss value will be continuously monitored during training, and the distribution of feature space will be analyzed by visualization. This will help verify whether facial features are successfully decoupled from illumination features and the extent to which this decorrelation improves recognition performance. The experimental results show that after introducing the light-face correlation loss training, the model's face recognition accuracy in low-light conditions is significantly improved, specifically, the recognition performance stability in different lighting environments is enhanced, and the recognition rate in low-light LFW is increased by about 10%. This result further confirms that by effectively reducing the correlation between features, the robustness of the model in practical applications can be significantly improved.

[0101] Through the implementation of this method, we not only optimized the feature representation, but also provided new ideas and directions for the future development of face recognition technology under low-light conditions.

[0102] Preferably, in step S2, the length of the feature set is set to set a length limit; (1) if the feature dimension exceeds the threshold, an improved feature selection algorithm is used for feature selection; (2) if the feature dimension is within the threshold range, a dynamically optimized deep learning algorithm is used for feature optimization.

[0103] The dynamic optimization deep learning algorithm aims to improve the effect of feature optimization by adaptively adjusting the learning rate and network parameters. The algorithm first extracts features from the training data and performs preliminary training using a neural network model. Then, by monitoring the loss function and model performance indicators, the learning rate is dynamically adjusted to ensure a rapid response to changes in the model's performance during training. In addition, the algorithm introduces regularization techniques to prevent overfitting and enhance the generalization ability of the model. In each training cycle, the algorithm evaluates the contribution of the current features and selects the most representative features by calculating feature importance. Finally, by iteratively updating the network parameters, the feature representation is optimized to ensure the best performance in a specific task. Overall, this dynamic optimization strategy makes the feature selection and optimization process more efficient and helps improve the accuracy of face recognition under low-light conditions.

[0104] Preferably, in step (1), an improved feature selection algorithm is adopted, and the improved formula is shown in the formula.

[0105] F new =αF old +(1-α)F input

[0106] Fold represents the features of the previous moment, and Fnew represents the input features of the current time period. α is a variable factor with a value range of [0,1], which is used to determine the degree of influence of the features of the previous time period on the current features.

[0107] Preferably, in step S3, when formulating the feature fusion scheme, the correlation between the illumination features and the facial features is fully considered, and the specific formula is shown in the formula.

[0108] R=β1G light +β2G face

[0109] Among them, β1 and β2 are two variable factors used to control the proportion of lighting features and facial features in the total feature benefits. Glight represents the gain of lighting features, Gface represents the gain of facial features, and R represents the total value benefit of the feature.

[0110] Preferably, in step S4, the feature fusion problem is transformed into an optimization problem of feature selection. The original problem makes a decision for each feature, 0 represents no selection, 1 represents selection, which is a 0-1 variable, and it is a constrained 0-1 programming problem, that is, an NP-hard problem. By considering the features that can maximize the recognition accuracy, the problem is transformed into a benefit maximization problem, and the problem is transformed into the following formula.

[0111]

[0112] The feature with the largest total gain is selected from the feature set for fusion, and xi is a 0-1 variable indicating whether the feature is selected.

[0113] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A low-light face recognition method based on feature decoupling and fusion, characterized in that: The following steps are involved: S1. Construct a triplet of face dataset, that is, take low-light face image as anchor, take low-light face image as positive sample, and normal brightness image as negative sample; shorten the feature space distance between anchor and positive, and increase the feature space distance between anchor and negative; S2. In the feature extraction stage, the image is processed by the feature extraction module to generate a 128-dimensional feature vector; the feature extraction module is used to extract the basic features of the face; S3, the feature vector then enters the decoupling module, which includes convolution Conv, batch normalization BN and ReLU activation function; the decoupling module is used to further optimize the feature representation and extract more representative facial features, thereby enhancing the model's ability to recognize subtle facial features; S4, the decoupled facial features are fed into the fusion module; the fusion module integrates the signals from different features through average pooling, linear transformation and sigmoid activation steps; this process generates the final facial feature representation; S5. The final facial features are input into the facial recognition module, which is responsible for classification and recognition. Different faces are accurately classified by inputting the extracted and fused features into the recognition module, thereby achieving facial recognition.

2. The low-light face recognition method based on feature decoupling and fusion according to claim 1, characterized in that: The step S1 constructs a triplet of face data set, specifically including: A1. Detect the key points of the face image through the face detection alignment network MTCNN. The face detection alignment network MTCNN refers to a multi-task convolutional neural network used to detect the face and its key points including eyes, nose, and mouth. MTCNN simultaneously performs face detection and key point positioning through a cascade structure, thereby achieving efficient and accurate face alignment. This network can effectively identify and locate facial features when processing images; and align the face image and crop it into an input image I of size 128×128; A2. Set up a low-light face pool and a normal-light face pool, randomly select a face image from the low-light face pool as an anchor, and then randomly select another low-light sample as a positive; randomly select one from the normal-light face pool as a negative; these three images will form a facial triplet based on illumination.

3. The low-light face recognition method based on feature decoupling and fusion according to claim 1, characterized in that: The feature extraction method in step S2 specifically includes: B1. Input the face image into the feature extractor Mobilefacenets, pass through an initial convolution layer to extract low-level features, and then transfer it to the depthwise separable convolution layer and the bottleneck block Bottleneck Blocks, which consists of two depthwise separable convolution layers connected by a linear transformation in the middle; B2. A global average pooling layer is used to reduce the spatial dimensions of each feature map, i.e., width W and height H, to 1×1, and the average value of each channel is generated to obtain a feature vector of fixed length.

4. The low-light face recognition method based on feature decoupling and fusion according to claim 1, characterized in that: The decoupling module in step S3 specifically includes: Given an input face image set Data = {low / dark, normal}, low / dark is a low-light face image set, normal is a normal-brightness face image set, and each image contains their identity ID; the purpose of the decoupling module is to separate independent illumination features and identity features, and there is no need to pay attention to the illumination label in this module; in the training stage, the decoupling module is trained by a given face image set; the goal of D-Net is to decouple the input image features into two components, and to separate the image illumination information from the facial information. This process is expressed as: Among them, I is the input face image, F is the decoupled 512-dimensional feature, when i=1, F is the decoupled 512-dimensional face feature, when i=2, F is the decoupled illumination feature, and D-Net represents the decoupling module; the input feature map passes through the continuous convolution layers in the two sub-modules F1 and F2 respectively to obtain rich abstract features, and then through the ReLU activation function, nonlinear processing is added to enhance the model's learning ability for facial features and illumination features, and finally a global average pooling layer is used to compress the features to unify the dimensions of the output features of the two sub-modules.

5. The low-light face recognition method based on feature decoupling and fusion according to claim 4, characterized in that: The feature fusion module in step S4 specifically includes the following steps: A1: Feature extraction: Extract residual facial features from illumination features, which can reflect the shape and texture of the face; use convolutional neural network (CNN) to process images under different illumination conditions to ensure that the extracted features have high discrimination; A2: Global pooling: Global pooling is performed on the extracted residual facial features to generate a feature vector of fixed size; A3: Importance weight learning: The globally pooled features are input into the fully connected layer, and the importance weights of each channel are learned through training. During this process, the model will continuously adjust the weights through the back-propagation algorithm to strengthen the expression of important features while suppressing noise and redundant information. A4: Feature separation and calibration: Separate the facial features from the illumination features based on the learned weights; then, use element-wise multiplication to fuse the calibrated illumination features with the initial facial features; A5: Optimization of fusion features: Through multiple iterations and optimization, the quality of fusion features is further improved; regularization technology is introduced to prevent overfitting and ensure the generalization ability of the model; This process is described as: Among them, X represents the input illumination feature map, Y represents the residual facial features in the illumination feature map, σ is a Sigmoid function, which is usually used to limit the output between 0 and 1, and W1 and W2 represent two linear transformations, W1 is used to reduce the number of channels, and W2 is used to restore the number of channels.

6. The low-light face recognition method based on feature decoupling and fusion according to claim 5, characterized in that: In step S5, the processed facial features are transmitted to the face matching module for subsequent recognition and matching, which specifically includes: The face matching module first receives the processed high-dimensional feature vector output from F-Net and compares it with the stored features in the face database; the face matching module uses a variety of similarity measurement methods to evaluate the match; in the matching process, the similarity score is calculated for the input feature vector and the feature vector of each sample in the database; then, through the pre-set similarity threshold, when the matching score reaches or exceeds the threshold, the current input face feature successfully matches a face sample in the database and returns the corresponding identity information; in the case of a match failure, a "no match result" prompt is returned, and the unrecognized features are recorded for subsequent analysis and processing; at the same time, it is dynamically updated based on the learned sample features to continuously improve the feature vectors in the database.

7. A low-light face recognition system based on feature decoupling and fusion, characterized in that: include: Face dataset triplet construction module: used to construct face dataset triplet, that is, low-light face image as anchor, low-light face image as positive sample, normal brightness image as negative sample; shorten the feature space distance between anchor and positive, and increase the feature space distance between anchor and negative; Feature extraction module: In the feature extraction stage, the image is processed by the feature extraction module to generate a 128-dimensional feature vector; the feature extraction module is used to extract the basic features of the face; Decoupling module: used for feature vector input decoupling module, which includes convolution Conv, batch normalization BN and ReLU activation function; the decoupling module is used to further optimize feature representation and extract more representative facial features, thereby enhancing the model's ability to recognize subtle facial features; Fusion module: used to feed the decoupled facial features into the fusion module; the fusion module integrates the signals from different features through average pooling AvgPool, linear transformation and Sigmoid activation steps; this process generates the final facial feature representation; Facial recognition module: The final facial features are input into the facial recognition module, which is responsible for classification and recognition; different faces are accurately classified by inputting the extracted and fused features into the recognition module, thereby achieving facial recognition.

Images