A face recognition method and device, electronic equipment and storage medium

Abstract

A face recognition method and device, electronic equipment and storage medium are related to the field of artificial intelligence. The method comprises: acquiring a face RGB image and a face IR image of a user for binocular live body verification; after verification, acquiring an illumination intensity; extracting a target face RGB feature and a target face IR feature; if the illumination intensity is greater than or equal to a first threshold, performing face RGB feature matching to obtain a plurality of first matching values; if the maximum first matching value is greater than or equal to a second threshold, face recognition is successful; if the maximum first matching value is less than the second threshold, the user is determined to be a stranger; if the illumination intensity is less than the first threshold, face IR feature matching is performed to obtain a plurality of second matching values; if the maximum second matching value is greater than or equal to a third threshold, face recognition is successful; if the maximum second matching value is less than the third threshold, the user is determined to be a stranger. The technical solution provided by the application can improve the accuracy of face recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to a face recognition method, device, electronic device, and storage medium. Background Technology

[0002] In recent years, facial recognition technology has been widely applied in security, finance, access control, and other fields. Facial recognition is a biometric-based intelligent identification method that automatically verifies or identifies an individual's identity by analyzing unique features (such as facial feature distribution, contours, and textures) in facial images or videos. Currently, most commonly used facial recognition methods extract features from RGB images and compare them to obtain the facial recognition result. However, RGB images are easily affected by lighting intensity, and relying solely on RGB image features for image comparison reduces the accuracy of facial recognition. Summary of the Invention

[0003] This application provides a face recognition method, device, electronic device, and storage medium, which can compare and recognize face RGB features or face IR features based on light intensity, thereby improving the accuracy of face recognition.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a face recognition method, the method comprising:

[0006] Acquire the user's RGB and IR facial images;

[0007] The user is subjected to binocular liveness verification based on the RGB image and IR image of the face;

[0008] After the binocular liveness verification is passed, the light intensity of the current environment is obtained;

[0009] Identify the target face RGB features corresponding to the face RGB image and the target face IR features corresponding to the face IR image, respectively;

[0010] If the light intensity is greater than or equal to the first threshold, then the target face RGB features are matched with each standard face RGB feature in the feature library to obtain multiple first matching values;

[0011] If the largest first matching value among the plurality of first matching values ​​is greater than or equal to the second threshold, then the standard face RGB feature corresponding to the largest first matching value is determined in the feature library, and the identity ID corresponding to the standard face RGB feature is determined as the user's identity ID;

[0012] If the largest first matching value among the plurality of first matching values ​​is less than the second threshold, then the user is determined to be a stranger;

[0013] If the light intensity is less than the first threshold, the target face IR feature is matched with each standard face IR feature in the feature library to obtain multiple second matching values;

[0014] If the largest second matching value among the plurality of second matching values ​​is greater than or equal to the third threshold, then the standard face IR feature corresponding to the largest second matching value is determined in the feature library, and the identity ID corresponding to the standard face IR feature is determined as the user's identity ID;

[0015] If the largest second matching value among the plurality of second matching values ​​is less than the third threshold, then the user is determined to be a stranger.

[0016] Secondly, embodiments of this application provide a face recognition device, comprising:

[0017] The image acquisition module is used to acquire the user's RGB image and IR image of the face;

[0018] The liveness verification module is used to perform binocular liveness verification on the user based on the face RGB image and the face IR image;

[0019] The feature extraction module is used to obtain the light intensity of the current environment after the binocular liveness verification is passed; and to identify the target face RGB features corresponding to the face RGB image and the target face IR features corresponding to the face IR image, respectively.

[0020] The feature comparison module is configured to: if the illumination intensity is greater than or equal to a first threshold, match the target face RGB features with standard face RGB features in a feature library to obtain multiple first matching values; if the largest first matching value among the multiple first matching values ​​is greater than or equal to a second threshold, determine the standard face RGB feature corresponding to the largest first matching value in the feature library, and determine the identity ID corresponding to the standard face RGB feature as the user's identity ID; if the largest first matching value among the multiple first matching values ​​is less than the second threshold, determine that the user is a stranger; if the illumination intensity is less than the first threshold, match the target face IR features with standard face IR features in a feature library to obtain multiple second matching values; if the largest second matching value among the multiple second matching values ​​is greater than or equal to a third threshold, determine the standard face IR feature corresponding to the largest second matching value in the feature library, and determine the identity ID corresponding to the standard face IR feature as the user's identity ID; if the largest second matching value among the multiple second matching values ​​is less than the third threshold, determine that the user is a stranger.

[0021] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform the above-described face recognition method.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed, implement the face recognition method provided in embodiments of this application.

[0023] By employing the above technical solution, the RGB and IR images of the face used for face recognition are first acquired, and binocular liveness verification is performed. After successful verification, the ambient light intensity is obtained, along with the RGB and IR features of the target face. The RGB or IR features are compared based on the light intensity to obtain multiple first and second matching values. The face recognition result is confirmed by comparing the largest first matching value with a second threshold or the largest second matching value with a third threshold. This light intensity-based face recognition method, by using different features for comparison depending on the scenario, effectively overcomes the adverse effects of complex lighting environments on the accuracy of feature extraction for recognition, improving the robustness of face recognition and thus enhancing its accuracy. Attached Figure Description

[0024] Figure 1This is a flowchart illustrating a face recognition method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0027] Explanation of reference numerals in the attached figures: 400, electronic device; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] In the description of the embodiments of this application, words such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0030] In the description of the embodiments of this application, the terms "first, second, third, and fourth" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, third, and fourth" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0031] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0032] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0033] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0034] In related technologies, face recognition results are obtained by performing single-modal feature comparison on RGB images. However, since face recognition is performed only on RGB images, and complex ambient lighting has a certain impact on the obtained RGB images, the accuracy of face recognition is reduced. To address the above problems, this application provides a face recognition method, device, electronic device, and storage medium that performs different feature comparisons for different lighting intensities, thereby further improving the accuracy of face recognition.

[0035] This invention discloses a face recognition method, such as... Figure 1 As shown, it includes the following steps:

[0036] Step S101: Obtain the user's RGB image and IR image of face.

[0037] Specifically, to acquire the user's RGB and IR facial images, a synchronously triggered RGB camera and infrared sensor (such as a FLIR thermal imager) or an integrated dual-modal industrial camera can be used for acquisition. A communication channel is established between the processor and the camera to transmit the images to the processor. The processor performs preliminary processing on the images, such as cropping and noise reduction. There are no restrictions on this process. Qualified images are selected as the user's RGB and IR facial images, providing a basis for subsequent binocular liveness verification and extraction of the target face's RGB and IR features.

[0038] Among them, RGB facial images refer to color images captured by visible light sensors, reflecting the surface visual features of the face. They help to identify traditional visual information such as facial shape and expression, and are suitable for scenarios with stable lighting (such as access control). IR facial images refer to thermal radiation images captured by infrared sensors, reflecting the temperature distribution and subcutaneous structure of the face. They help overcome visible light interference (such as in dark environments or strong backlighting), enhance liveness detection capabilities, and are suitable for high-security scenarios.

[0039] Step S102: Perform binocular liveness verification on the user based on the face RGB image and face IR image.

[0040] Specifically, face detection algorithms, face quality algorithms, and face landmark algorithms can be used to collaboratively verify RGB and IR images of the face to obtain binocular liveness verification results. This dual-modal verification ensures the stability of verification results under complex lighting conditions.

[0041] Binocular liveness verification refers to the verification result of determining whether a face is a real living person (rather than a non-living result such as a photo, video, or mask) by simultaneously analyzing RGB and IR images.

[0042] Based on the above embodiments, as an optional embodiment, for Figure 1 Step S102 shown can be implemented through the following steps, which are explained in detail below.

[0043] Face detection is performed on the RGB and IR images of the face. If no face is detected in the RGB and / or IR images of the face, the binocular liveness verification is deemed to have failed.

[0044] Specifically, to detect faces in RGB images, deep learning-based face detection algorithms can be used, such as multi-task cascaded convolutional neural network models (MTCNN) and single-stage object detection algorithms (YOLO). No specific limitations are specified here. The trained model scans and analyzes the face RGB image in the processor to obtain the detection result. To detect faces in IR images, a detection model optimized for infrared images can be used. The processor utilizes the radiation difference features between the face and the background in the infrared image to identify and determine the detection result.

[0045] Specifically, after obtaining the detection results of the face RGB image and face IR image, if at least one of the detection results fails to detect a face, a preliminary binocular liveness verification result will be directly output.

[0046] This method enables rapid determination of face presence in the initial stage of binocular liveness verification, effectively filtering out cases where subsequent verification is impossible due to the absence of a captured face. This improves the efficiency of the verification process, avoids ineffective feature extraction and analysis, and lays a solid foundation for subsequent liveness verification, ensuring that only images containing valid facial information proceed to the next stage, thus further enhancing the reliability and practicality of binocular liveness verification.

[0047] For example, the obtained RGB image of user F1's face did not detect face 1, the IR image of user F2's face did not detect a face, and neither the RGB image nor the IR image of user F3's face detected a face. Therefore, none of the above three users passed the binocular liveness verification.

[0048] If a face is detected in both the RGB and IR images of a face, then the face in the RGB image is subjected to contrast detection, blur detection, and skin color region integrity detection to obtain the first detection result, and the face in the IR image is subjected to infrared radiation intensity distribution detection to obtain the second detection result.

[0049] Specifically, to obtain the first detection result, the contrast of the RGB image of the face can be detected by calculating the image grayscale histogram, analyzing the brightness difference between the face region and the background, and comparing it with standard parameters to obtain a contrast score. Blur detection can be performed by using edge detection algorithms (such as the Laplacian operator) to calculate image sharpness, comparing it with threshold data to obtain a sharpness score. Skin color region integrity detection can utilize the range characteristics of skin color in the RGB space, determining the skin color region through pixel analysis, calculating its coverage, and comparing it with standard parameters to obtain an integrity score. To obtain the second detection result, the infrared radiation intensity distribution of the IR image of the face can be detected using infrared image analysis algorithms, extracting the radiation intensity values ​​of each region, comparing them with normal range data, and analyzing whether the distribution conforms to a pattern. These detection processes can be implemented using computer vision software, automatically completing calculations and judgments based on preset parameters and thresholds.

[0050] The first detection result refers to the index of the face quality algorithm performed on the RGB image of the face, including contrast detection, blur detection, and skin color region integrity detection. This can be expressed as a contrast score, sharpness score, and integrity score. The second detection result refers to the index of whether the infrared radiation intensity distribution in the IR image of the face conforms to the law of liveness.

[0051] The standard parameters used for contrast testing are reference data used to measure whether the contrast of the face region in an RGB image of a face is acceptable. These mainly include the range of brightness difference between the face region and the surrounding background, the brightness contrast standard of different parts of the face (such as facial features and skin), the contrast adjustment coefficient under different lighting conditions, and the contrast adaptation parameters for different skin tones.

[0052] The threshold data used for blur detection are the critical data for judging whether a face is clear in an RGB image. It mainly includes the image sharpness score threshold (such as the score calculated by edge sharpness), the grayscale change threshold between pixels, the blur threshold correction value under different image resolutions, and special judgment thresholds for motion blur, etc.

[0053] The standard parameters used for skin color region integrity detection are reference data for detecting the integrity of skin color regions in RGB images of faces. These mainly include the range of normal skin color in the RGB color space, the standard facial proportions that the skin color region should cover, the texture features of the skin color region, the differences in skin color features among different races, and correction parameters for skin color affected by lighting.

[0054] Normal range data used for infrared radiation intensity detection is standard data for determining whether the distribution of infrared radiation intensity in a face IR image conforms to the characteristics of a living body. It mainly includes the normal infrared radiation intensity range of various areas of the face (such as the forehead, cheeks, lips, etc.), the overall distribution pattern of facial infrared radiation, the range of infrared radiation intensity fluctuations under different physiological states (such as calm, after exercise), and normal range adjustment data under different ambient temperatures.

[0055] Through the above embodiments, after detecting a face, further in-depth detection of the face quality in both RGB and IR images can be performed. This ensures the integrity and clarity of the image information used for subsequent verification, and also increases the basis for judging liveness verification through multi-dimensional detection. This multi-index, multi-modal detection method effectively improves the ability to identify forgery methods, providing a stronger guarantee for the accuracy and reliability of binocular liveness verification.

[0056] If the first test result and / or the second test result fail, the binocular liveness verification is deemed unsuccessful.

[0057] Specifically, three threshold values ​​are set for the first test result. If at least one indicator fails to meet the threshold, the first test result is deemed unsuccessful. If the first test result fails but the second test result passes, the binocular liveness verification is deemed unsuccessful; if the first test result passes but the second test result fails, the binocular liveness verification is deemed unsuccessful; if both the first and second test results fail, the binocular liveness verification is deemed unsuccessful.

[0058] For example, assuming a contrast score of 0.85 is acceptable, a sharpness score of 0.8 is acceptable, and an integrity score of 0.88 is acceptable. If user F4's RGB and IR images of their face detect a face, their first detection results are a contrast score of 0.89, a sharpness score of 0.72, and an integrity score of 0.89. Their second detection result is "pass." Therefore, user F4's first detection result is "fail," and the binocular liveness verification result is "failed," and subsequent steps are not performed.

[0059] If both the first and second detection results pass, the key points of the face in the face RGB image and face IR image are extracted, and prompts are output to prompt the user to complete the dynamic action.

[0060] Specifically, after both the first and second detection results are passed, to extract facial key points from the RGB and IR images of the face, deep learning-based facial key point detection algorithms (such as Dlib, improved MTCNN models, etc.) can be used. The trained model parameters are then used in the processor to process the image, locate, and output the coordinates of each key point. A dynamic action library can be constructed by collecting a large number of live dynamic action samples. The prompt template is designed with standardized text based on the characteristics of the actions. When generating instructions, actions are randomly selected from the action library and used to populate the template variables. Finally, the extracted key point data and the generated prompt instructions are output through a network interface.

[0061] Through the above embodiments, based on successful preliminary detection, key facial features can be accurately extracted and clear dynamic action prompts can be generated. The extraction of key features provides a precise feature base for subsequent tracking and verification of dynamic actions, while the introduction of dynamic actions further enhances the security of liveness verification, effectively preventing static forgery methods such as photos and videos.

[0062] Among them, the facial landmark extraction model parameters are the set of key parameters that support the operation of the facial landmark extraction model. They mainly include the weight parameters, feature mapping parameters, adaptation parameters for different facial poses and expressions, and related parameters for model iteration and optimization, etc., used by the model to locate key facial features (such as the corners of the eyes, the tip of the nose, the corners of the mouth, etc.).

[0063] The dynamic action library is a pre-collected set of dynamic facial actions used for liveness verification. The prompt template is a standardized prompt text format corresponding to each dynamic action, which mainly includes the action name (such as blinking, turning the head, opening the mouth, etc.), the standard description of the action, as well as variables in the prompt template (such as the number of times the action is performed, the duration, etc.), dynamic actions of different difficulty levels, and prompt language styles suitable for different scenarios.

[0064] For example, if both the first and second detection results for user F5 pass, then the key points of user F5's RGB and IR face images are extracted, and a prompt command to make user F5 open their mouth is output.

[0065] Based on dynamic motion, key points are tracked across frames to obtain motion trajectories.

[0066] Specifically, to achieve cross-frame keypoint tracking, for face RGB images and face IR images, starting from the initial keypoint coordinates, the processor searches for the most similar keypoints in consecutive frames as the tracking result, and can also optimize the keypoint tracking algorithm parameters. During the tracking process, face RGB image frames and face IR image frames are synchronized through dynamic motion time series data to generate motion trajectories, including keypoint coordinates and timestamps for each frame.

[0067] Among them, the key point tracking algorithm parameters are the key parameter settings that support the operation of cross-frame tracking algorithms. They mainly include the similarity threshold for key point matching, the parameters of the motion prediction model, the threshold for occlusion handling, the tracking parameter adjustment coefficients for different dynamic actions (such as fast actions and slow actions), and the smoothing parameters to deal with image noise.

[0068] Motion trajectory refers to the sequence data describing the positional changes of a single keypoint in consecutive image frames. It is a continuous trajectory curve formed by the coordinates of the keypoint in each frame and the corresponding timestamp, reflecting the motion pattern of the keypoint.

[0069] Through the above embodiments, it is possible to achieve cross-frame tracking of key points and obtain motion trajectories based on dynamic actions, making full use of continuous frame data and multimodal information, thereby improving the accuracy and completeness of motion trajectories.

[0070] Based on the motion trajectory, the continuity of dynamic actions is determined.

[0071] Specifically, speed smoothness detection, path rationality detection, and temporal consistency detection can be performed based on the motion trajectory. Combining human kinematics and deep machine learning models, such as training a Long Short-Term Memory (LSTM) network model to learn the trajectory differences between real and fake actions, thresholds are set for multi-dimensional indicators to obtain the corresponding speed smoothness detection results, path rationality detection results, and temporal consistency detection results. No specific restrictions are imposed here.

[0072] Among these, speed smoothness refers to the continuous change in acceleration of a real movement without abrupt changes. Path rationality refers to whether the trajectory of the movement conforms to physiological laws; for example, the trajectory of blinking should be arc-shaped, not a straight line. Temporal consistency refers to the time proportion of each stage of the movement (such as raising a hand, pausing, and lowering it) conforming to the laws of human kinematics.

[0073] Through the above embodiments, the continuity of dynamic actions is judged based on the motion trajectory, making full use of the temporal and spatial characteristics of the trajectory data, and can objectively quantify the degree of continuity of the action.

[0074] If the continuity is not up to standard, the binocular liveness verification is deemed unsuccessful.

[0075] Specifically, the results of speed smoothness detection, path rationality detection, and timing consistency detection in the above steps are combined for judgment. If at least one of the detection results is unqualified, the binocular liveness verification is deemed to have failed.

[0076] For example, assuming the threshold for passing speed smoothness is 0.9, the threshold for passing path rationality is 0.85, and the threshold for passing timing consistency is 0.75, if the speed smoothness test result for the user's F5 prompt command is 0.75, the path rationality test result is 0.85, and the timing consistency test result is 0.8, then the speed smoothness result fails, the continuity is deemed unqualified, and the binocular liveness verification fails.

[0077] If the continuity is satisfactory, the binocular liveness verification is considered successful.

[0078] For example, if user F6 passes all the tests in step S102, and the speed smoothness test result for the prompt command is 0.95, the path rationality test result is 0.89, and the timing consistency test result is 0.9, then the continuity is qualified and the binocular liveness verification is passed.

[0079] Step S103: After the binocular liveness verification is passed, obtain the light intensity of the current environment.

[0080] Specifically, a dedicated light detection device (such as a light sensor) can be used to read the light measurement value through the device interface and obtain the current temperature through a temperature sensor. The light intensity can then be corrected using a reference temperature, a first temperature correction coefficient, and a second temperature correction coefficient, as shown in Formula 1.

[0081] (1)

[0082] Where I represents the current ambient light intensity, This represents the illumination measurement value collected by the detection equipment. It is the first temperature correction factor. This is the second temperature correction factor, where T is the current temperature obtained. Indicates the reference temperature.

[0083] For example, assume a first temperature correction factor. The second temperature correction factor is 0.0025 / ℃. The value is 0.000015 / ℃, the current temperature T is 32℃, and the reference temperature is... At 25℃, the measured value of light intensity If the illuminance is 500 lux, then the temperature-corrected illuminance I is 500 * [1 + 0.0025 * 7 + (0.000015 * 7)]. 2]≈508.75 lux.

[0084] Step S104: Identify the target face RGB features corresponding to the face RGB image and the target face IR features corresponding to the face IR image.

[0085] Specifically, based on the acquired RGB and IR images of the face, the processor identifies the RGB and IR features of the target face. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), or pre-trained ResNet residual network models or ArcFace face recognition models can be used to extract the RGB features of the target face. The IR features of the target face can be extracted based on thermal radiation statistics (such as regional mean and variance) or by training a lightweight convolutional neural network model (CNN).

[0086] Through the above embodiments, the RGB features and IR features of the target face are identified respectively for subsequent identification and comparison operations. Face recognition based on multiple features can effectively improve the accuracy of face recognition.

[0087] Based on the above embodiments, as an optional embodiment, for Figure 1 Step S104 shown can be implemented through the following steps, which are explained in detail below.

[0088] Noise in the RGB image of a face is removed by bilateral filtering, and the facial texture of the denoised RGB image is obtained by histogram equalization, resulting in an enhanced RGB image of the face. The ROI region containing the complete face is then extracted from the enhanced RGB image of the face.

[0089] Specifically, the spatial domain standard deviation, grayscale domain standard deviation, and filter window size can be set in the processor to process the RGB image of the face. By calculating the spatial distance and grayscale similarity between pixels, noise removal and edge preservation are achieved. The pixel value range and equalization intensity coefficient are set to perform global histogram equalization on the denoised RGB image of the face, enhancing texture contrast by adjusting the pixel grayscale distribution. When cropping the Region of Interest (ROI), key points such as the corners of the eyes, the tip of the nose, and the tip of the chin are first located. Using the midpoint of the line connecting the eyes as the upper reference, the tip of the chin as the lower boundary, and the widest part of the cheeks as the boundaries on both sides, the region is cropped according to a certain aspect ratio to ensure that the complete face is included. The extraction of the ROI not only improves image quality but also highlights facial information, laying a high-quality image foundation for subsequent feature extraction.

[0090] In this context, the Region of Interest (ROI) is a specific area in images, videos, or other visual data that is manually or automatically defined and requires focused analysis or processing. Its core function is to focus on the parts of the data that have practical significance or research value, eliminating irrelevant information to improve processing efficiency, reduce computational costs, or enhance analytical accuracy.

[0091] The ROI region is identified to obtain 128-dimensional LBP texture low-level features, 256-dimensional HOG shape mid-level features, and 128-dimensional depth convolution high-level features, which are used as the RGB features of the target face.

[0092] Specifically, the neighborhood radius and number of sampling points can be set, and uniformity and rotation invariance processing can be used to obtain 128-dimensional LBP texture low-level features. The distribution of texture patterns in different regions can then be statistically analyzed. Alternatively, the image can be divided into cell units, with cell units of a certain size forming blocks. The number of gradient directions can be set, and the gradient direction histogram of each cell unit can be calculated first. Then, the features within each block can be normalized. By appropriately dividing the image into blocks and setting the number of cell units, 256-dimensional HOG shape mid-level features can be obtained. When extracting 128-dimensional deep convolutional high-level features, a pre-trained lightweight convolutional neural network model can be used. The output of the final fully connected layer of the network can be adjusted to 128 dimensions. If necessary, the model can be fine-tuned using a target face dataset to improve the specificity of the features.

[0093] Here, LBP texture low-level features refer to 128-dimensional features based on local binary patterns extracted from the ROI region image, used to describe the microscopic features of local facial texture. These include binary patterns formed by comparing the gray values ​​of each pixel in the image with its neighboring pixels, the distribution frequency of these patterns in different areas of the face, and the differences in texture patterns at different scales (such as pore texture at a small scale and wrinkle texture at a large scale), etc., without specific limitations here.

[0094] HOG (Heterogeneous Oriented Gradient) mid-level features refer to 256-dimensional features based on gradient orientation histograms extracted from the ROI (Region of Interest) image, used to characterize the contour and shape features of the face. Typically, the image is divided into multiple cell units, the gradient direction and intensity of pixels within each unit are calculated, the resulting histogram is statistically analyzed, and then adjacent cell units are grouped together for feature normalization.

[0095] Deep convolutional high-level features refer to 128-dimensional abstract features extracted from Region of Interest (ROI) images using deep convolutional neural networks. These features comprehensively reflect the complex semantic information of the face. They are high-dimensional feature vectors obtained after multiple layers of convolution, pooling, activation, and other operations, containing latent information such as facial identity, expression, and pose.

[0096] The thermal noise points of the face IR image are removed by morphological filtering, and the pixel values ​​of the face IR image with thermal noise removed are mapped to a fixed range by temperature normalization to highlight the facial temperature difference of the face IR image, thus obtaining a facial temperature difference image.

[0097] Specifically, for thermal noise points in facial IR images, rectangular structural elements of a certain size are constructed. An opening operation is then performed: first, erosion is applied to remove small bright noise points, followed by dilation to restore the main contours and temperature features of the image, resulting in a denoised image. For temperature normalization, the original temperature range of the denoised image is first statistically analyzed, and a target mapping range is set. Through linear mapping, the temperature value of each pixel is mapped to a fixed range, as shown in Equation 2, generating a facial temperature difference image that highlights facial temperature differences. This ensures image cleanliness and the integrity of temperature features, providing a high-quality image foundation for subsequent identification of temperature distribution and thermal gradient features from facial temperature difference images.

[0098] (2)

[0099] Among them, G i T represents the target value obtained for the i-th pixel, ranging from 0 to N. i T is the original temperature value of the i-th pixel. min This indicates the lowest temperature at which the face was detected, T. max This indicates the highest temperature detected for that face.

[0100] For example, N is set to 255, and the lowest temperature T at which user F6's face is detected is... min The highest temperature was 32.1℃, and the highest temperature was T. max The initial temperature value (T7) is 37.8℃, and the original temperature value (T7) is 36.1℃. Therefore, the target value (G7) is... .

[0101] The temperature distribution characteristics and thermal gradient characteristics of the facial temperature difference image are identified as the IR features of the target face.

[0102] Specifically, based on temperature zone division thresholds (e.g., high temperature threshold 200-255, low temperature threshold 0-50, normal temperature threshold 51-199), high temperature zone, low temperature zone, and normal temperature zone are identified in the facial temperature difference image, and their location coordinates and area proportions are recorded. A sliding window is used to calculate the mean, extreme values, and variance of the temperature in each region, and a fixed-size window is set to obtain the temperature distribution characteristics. The Sobel operator of a fixed size is used to calculate the horizontal and vertical gradients of the image, and the gradient magnitude and direction are obtained through gradient synthesis. The distribution range of gradient magnitude and the main orientation of gradient direction are statistically analyzed to locate areas with significant gradient changes, thereby obtaining thermal gradient characteristics.

[0103] Among them, the temperature distribution feature is extracted from the facial temperature difference image, reflecting the spatial distribution of facial temperature. The thermal gradient feature is extracted from the facial temperature difference image, reflecting the rate and direction of facial temperature change.

[0104] Through the above embodiments, temperature distribution features and thermal gradient features are identified from facial temperature difference images to form a target face IR feature set, fully utilizing the advantage of infrared images in reflecting temperature information. Temperature distribution features depict the spatial distribution pattern of facial temperature, while thermal gradient features reflect the trend of temperature change. The two complement each other and can comprehensively and accurately describe the infrared characteristics of the target face.

[0105] Step S105: If the light intensity is greater than or equal to the first threshold, then the target face RGB features are matched with the standard face RGB features in the feature library to obtain multiple first matching values.

[0106] The first threshold is used to distinguish between light intensity and light intensity, so as to perform subsequent feature comparison operations; the feature library refers to the base library of faces with recognition permissions collected and built in the processor for recognition when performing actual face recognition applications; the first matching value is the key data used to determine the face matching result.

[0107] For example, if the ambient light intensity at the time of user F6 recognition is 508.75 lux, and assuming the first threshold is 300 lux, then the RGB features of the face of user F6 are compared, and multiple first matching values ​​are calculated.

[0108] against Figure 1 The step S105 shown, as an optional embodiment, can be implemented through the following steps, which are described in detail below.

[0109] Obtain the 512-dimensional target feature vector corresponding to the RGB features of the target face. The 512-dimensional target feature vector includes a 128-dimensional target LBP texture bottom layer feature vector, a 256-dimensional target HOG shape middle layer feature vector, and a 128-dimensional target depth convolution high layer feature vector.

[0110] Specifically, by acquiring the 128-dimensional LBP texture low-level features, 256-dimensional HOG shape mid-level features, and 128-dimensional depth convolution high-level features of the target face RGB features in the processor, a 512-dimensional target feature vector is obtained.

[0111] Obtain the 512-dimensional standard feature vector corresponding to the standard face RGB features. The 512-dimensional standard feature vector includes a 128-dimensional standard LBP texture bottom layer feature vector, a 256-dimensional standard HOG shape middle layer feature vector, and a 128-dimensional standard depth convolution high layer feature vector.

[0112] Specifically, by obtaining the vectors corresponding to the 128-dimensional LBP texture low-level features, the 256-dimensional HOG shape mid-level features, and the 128-dimensional depth convolution high-level features of the standard face RGB features in the feature library in the processor, a 512-dimensional standard feature vector is obtained.

[0113] Multiply the 512-dimensional target feature vector with the 512-dimensional standard feature vector for each dimension to obtain the feature matching value corresponding to each dimension of the target face RGB feature.

[0114] Specifically, the 512-dimensional target feature vector and the 512-dimensional standard feature vector can be matched one-to-one based on key point data, and the product of the feature vectors of each dimension after matching is used as the feature matching value for each dimension. By calculating the feature matching value through multiplication of the corresponding dimensions, the matching details between the target and the standard features in each dimension can be accurately captured, overcoming the limitation of the comprehensive matching value masking the dimensional differences, and providing a more refined basis for feature analysis.

[0115] The arithmetic mean of the feature matching values ​​of each dimension corresponding to the 128-dimensional target LBP texture bottom layer features, the 256-dimensional target HOG shape middle layer features, and the 128-dimensional target depth convolution high layer features are calculated respectively to obtain the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value.

[0116] Specifically, the feature matching values ​​obtained in the above steps are matched one-to-one with the 128-dimensional LBP texture bottom layer features, the 256-dimensional HOG shape middle layer features, and the 128-dimensional depth convolution high layer features, and the arithmetic mean is calculated to obtain the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value, as shown in Formula 3.

[0117] ,

[0118] ,

[0119] (3)

[0120] Among them, S low S represents the underlying feature matching value. mid Shigh represents the mid-level feature matching value, and Shigh represents the high-level feature matching value. This represents the i-th dimension of the target feature vector. Let represent the i-th dimensional standard eigenvector.

[0121] If the light intensity is greater than or equal to the first threshold and less than the fourth threshold, then the first weight is assigned to the bottom-level feature matching value, the middle-level feature matching value, and the high-level feature matching value.

[0122] Specifically, the fourth threshold is greater than the first threshold and is used to further differentiate illumination intensity. The first weight refers to the weight ratio of the bottom-level feature matching value, the middle-level feature matching value, and the high-level feature matching value in the first matching value, and can be set according to actual conditions. Through the dynamic weight allocation strategy of illumination range, the adaptability and accuracy of face recognition under different illumination conditions are improved, providing effective support for identity verification in complex environments.

[0123] Based on the first weight, the matching values ​​of the bottom layer feature, the matching values ​​of the middle layer feature, and the matching values ​​of the top layer feature are weighted and summed to obtain the first matching value.

[0124] Specifically, based on the first weight pre-set in the processor, the calculated low-level feature matching value, mid-level feature matching value, and high-level feature matching value are weighted and summed, as shown in Formula 4, to obtain the first matching value. Through weighted matching, the overall feature matching value is quantified, effectively improving the accuracy of face recognition within this lighting range and providing a reliable basis for identity verification.

[0125] S1= *S low + *S mid + *S high (4)

[0126] Where S1 represents the first matching value, , , This indicates the matching value S of the underlying features. low Mid-layer feature matching value S mid and high-level feature matching value S high The first weight assigned.

[0127] For example, the fourth threshold is set to 1000 lux, the first threshold is 300 lux, and the light intensity during user F6 recognition is 508.75 lux. The matching value S with the underlying feature value of a certain standard face RGB feature is... low The mid-level feature matching value S is 0.95. mid The high-level feature matching value S is 0.88. high If the value is 0.92, then assign it the first weight and set the first weight. 0.3 0.3 If the value is 0.4, then the corresponding first matching value S1 is 0.95*0.3+0.88*0.3+0.92*0.4=0.917.

[0128] If the light intensity is greater than or equal to the fourth threshold, the weight ratio of the high-level feature matching value is increased, and the second weight is redistributed to the low-level feature matching value, the middle-level feature matching value, and the high-level feature matching value.

[0129] Specifically, when the light intensity obtained by the processor exceeds the fourth threshold, it indicates that the environment is too bright. In this case, the weights of the low-level, mid-level, and high-level feature matching values ​​need to be reallocated as a second weight, with the weight proportion corresponding to the high-level feature matching values ​​needing to be increased. By reallocating the weights, the interference of strong light environments on low-level and mid-level features can be specifically addressed. By dynamically adjusting the weights, the high-level features of the deep convolution can play a dominant role in the matching process, fully utilizing their stability under strong light, thereby improving the accuracy of the overall matching results.

[0130] Based on the second weight, the matching values ​​of the bottom layer feature, the matching values ​​of the middle layer feature, and the matching values ​​of the top layer feature are weighted and summed to obtain the first matching value.

[0131] Specifically, referring to Formula 4, the weighted sum of the bottom-level feature matching value, the middle-level feature matching value, and the high-level feature matching value is used to obtain the first matching value under strong light conditions.

[0132] For example, user F7 was under illumination of 1200 lux during recognition, and the matching value S of the underlying feature of a standard face RGB feature was... low The mid-level feature matching value S is 0.75. mid The high-level feature matching value S is 0.65. high If the value is 0.53, then assign it the first weight and set the first weight. 0.2 0.2 If the value is 0.6, then the corresponding first matching value S1 is 0.75*0.2+0.65*0.2+0.53*0.6=0.598.

[0133] Step S106: If the largest first matching value among multiple first matching values ​​is greater than or equal to the second threshold, then determine the standard face RGB feature corresponding to the largest first matching value in the feature library, and determine the identity ID corresponding to the standard face RGB feature as the user's identity ID.

[0134] Specifically, after obtaining multiple first matching values, they are sorted, and the largest first matching value is taken for operation. If the largest first matching value is greater than or equal to the second threshold, it is determined that the user has recognition authority, and the identity ID corresponding to the standard face RGB feature of the largest first matching value is identified as the user's identity ID.

[0135] The second threshold is used to determine the threshold corresponding to the first batch of configurations obtained from the face RGB feature comparison, and can be set according to the actual situation.

[0136] For example, if the second threshold is set to 0.9, and the maximum first matching value of user F6 is 0.96, and 0.96 > 0.9, then user F6 is deemed to have recognition authority. If the identity ID corresponding to the standard face RGB feature corresponding to the maximum first matching value is determined to be 0028, then user F6 is deemed to be user 0028.

[0137] Step S107: If the largest first matching value among multiple first matching values ​​is less than the second threshold, then the user is determined to be a stranger.

[0138] Specifically, when the maximum first matching value obtained is less than the second threshold, it is determined that the user does not have the identification authority and the user is identified as a stranger.

[0139] For example, the maximum first match value for user F7 is 0.66. Since 0.66 < 0.9, user F7 is deemed to lack identification authority and is identified as a stranger.

[0140] Step S108: If the illumination intensity is less than the first threshold, the target face IR features are matched with the standard face IR features in the feature library to obtain multiple second matching values.

[0141] Specifically, when the processor detects that the current ambient light intensity is less than a first threshold, it initiates a face IR feature matching process. All standard face IR features are retrieved from the feature library (each standard feature includes temperature distribution features and thermal gradient features). The temperature distribution features of the target and the standards are converted into vectors (e.g., composed of indicators such as the proportion of high-temperature areas and the average temperature). The thermal gradient features are also converted into vectors (e.g., composed of indicators such as the average gradient magnitude and the proportion of main directions). Multiple second matching values ​​are obtained through vector matching. Face IR features are less affected by visible light, and their temperature distribution and thermal gradient features exhibit unique individual differences, effectively compensating for the deficiencies of face RGB features in insufficient lighting conditions.

[0142] Based on the above embodiments, as an optional embodiment, for Figure 1 Step S108 shown can be implemented through the following steps, which are explained in detail below.

[0143] Obtain the target absolute temperature component of the IR features of the target face, and the standard absolute temperature component of the IR features of the standard face.

[0144] Specifically, based on the temperature distribution features obtained in step S104, the recognized facial region can be divided into 16 regions (such as the forehead, nose tip, left and right cheeks, etc.) in the processor. Each point corresponds to 4 dimensions (mean temperature, maximum temperature, minimum temperature, and variance), resulting in 64-dimensional target absolute temperature components. The extraction process of the standard absolute temperature components can refer to the embodiment of step S104, and they can correspond one-to-one with the target absolute temperature components. The unified extraction parameters and region division standards ensure the comparability and matching accuracy of the target and standard absolute temperature components.

[0145] The standard absolute temperature component serves as the absolute temperature feature component of each facial region, acting as a comparison benchmark. It is stored in a feature library and provides a matching reference for the target absolute temperature component. The target absolute temperature component reflects the unique physiological temperature characteristics of the target face and is crucial foundational data in the target face's IR features.

[0146] By calculating the target absolute temperature component and the standard absolute temperature component, the Euclidean distance between the target face IR features and the standard face IR features is obtained.

[0147] Specifically, based on the extracted 64-dimensional target absolute temperature components and 64-dimensional standard absolute temperature components, the target absolute temperature components and standard absolute temperature components are mapped one-to-one, as shown in Formula 5, and the Euclidean distance is calculated.

[0148] D1 = (5)

[0149] Where D1 represents the Euclidean distance between the target face IR features and the standard face IR features, x i Let y represent the i-th dimension of the target's absolute temperature component. i Let represent the i-th dimension of the standard absolute temperature component, and let the Euclidean distance D1 range from [1, 10].

[0150] For example, if the illumination intensity for user F8 during recognition is 260 lux < 300 lux, then facial IR feature comparison is performed on user F8. If user F8's facial IR feature is compared with a certain standard facial IR feature... If the value is 64, then the Euclidean distance D1 between user F8 and the standard face IR feature is... =8.

[0151] The Euclidean distance is normalized to obtain the absolute temperature matching value of the target face IR feature.

[0152] Specifically, the Euclidean distance obtained above is normalized so that its target value ranges from [0, 1], as shown in Formula 6, and is used as the absolute temperature matching value between the target face IR feature and a certain standard face IR feature in the special database. Through the above embodiment, the absolute temperature matching value is obtained by normalizing the Euclidean distance, transforming the distance index that originally reflected the difference into an intuitive matching degree index, thereby improving the interpretability and practicality of the matching results of the absolute temperature component of the target face IR feature.

[0153] D = (6)

[0154] Where D represents the absolute temperature matching value between the target face IR feature and a certain standard face IR feature, and the range is [0, 1].

[0155] For example, if user F8 has an Euclidean distance D1=8 with a certain standard face IR feature during recognition, then its absolute temperature matching value D with that standard face IR feature is 1-8 / 10=0.2.

[0156] Obtain the target thermal gradient component of the IR features of the target face and the standard thermal gradient component of the IR features of the standard face.

[0157] Specifically, based on the thermal gradient features obtained in step S104, the processor summarizes their gradient directions, including horizontal and vertical directions. Each gradient direction corresponds to 32 dimensions, forming a 64-dimensional target thermal gradient component. The extraction process for the standard thermal gradient component is consistent with that for the target thermal gradient component, allowing for a one-to-one correspondence with the 64-dimensional target thermal gradient component. By extracting the 64-dimensional thermal gradient component, the feature dimensions are simplified while ensuring that key thermal gradient information is not lost, thus improving the efficiency of matching calculations and the convenience of storage.

[0158] The target thermal gradient component reflects facial thermal gradient features, showcasing the rate and direction of temperature changes on the target face. It is a crucial component in the target face's IR features, reflecting dynamic temperature changes. The standard thermal gradient component refers to the reference component stored in the feature library as a comparison standard, providing a reliable basis for identity recognition and improving the accuracy of matching results.

[0159] Calculate the cosine similarity between the target thermal gradient component and the standard thermal gradient component, and use it as the thermal gradient matching value of the target face IR feature.

[0160] Specifically, based on the 64-dimensional target thermal gradient components and the 64-dimensional standard thermal gradient components obtained in the processor, the thermal gradient matching value between the target face IR features and a certain standard face IR feature in the feature library is obtained through the cosine similarity algorithm, as shown in Equation 7. Through the above embodiments, the similarity between the target and the standard face in thermal gradient features can be accurately quantified, making full use of the distribution information of high-dimensional features.

[0161] H = (7)

[0162] Where H represents the thermal gradient matching value between the target face IR feature and a standard face IR feature in the feature library, and a i Let b represent the i-th dimension of the target thermal gradient component. i This represents the i-th dimension of the standard thermal gradient component.

[0163] For example, when user F8 compares a face against a standard facial IR feature, It is 100. It is 100. If the value is 144, then the thermal gradient matching value H with the standard face IR feature is 100 / * ≈0.83.

[0164] A third weight is assigned to the absolute temperature matching value and the thermal gradient matching value.

[0165] Specifically, the third weight refers to the weight ratio of the absolute temperature matching value and the thermal gradient matching value in the second matching value. It can be set in advance in the processor according to the actual situation. There are no specific restrictions here. Based on the stability advantage of thermal gradient features in temperature change scenarios, the weight of thermal gradient matching value is higher than that of absolute temperature matching value when allocating the third weight, so as to improve the reliability of comprehensive matching.

[0166] Through the above embodiments, setting a third weight takes into account both the basic reference value of absolute temperature and the pattern differentiation value of thermal gradient, while avoiding the limitations of single feature matching, thus improving the accuracy and robustness of face IR feature comparison.

[0167] Based on the third weight, the absolute temperature matching value and the thermal gradient matching value are weighted and summed to obtain the second matching value.

[0168] Specifically, based on a pre-defined third weight in the processor, the processed absolute temperature matching value and thermal gradient matching value are weighted and summed, as shown in Equation 8, to obtain the second matching value. This calculation method is efficient and intuitive, providing accurate quantitative basis for identity recognition in low-light environments.

[0169] S2= *D + *H (8)

[0170] Where S2 represents the second matching value between the target face IR feature and a certain standard face IR feature. , This represents the third weight assigned to the absolute temperature feature matching value and the thermal gradient feature matching value.

[0171] For example, assuming a third weight It is 0.45. The absolute temperature matching value D obtained by user F8 in comparison with a certain standard face IR feature is 0.6, and the thermal gradient matching value H is 0.83. Then the second matching value S2 of user F8 with the standard face IR feature is 0.45*0.6+0.55*0.83=0.7265.

[0172] Step S109: If the largest second matching value among multiple second matching values ​​is greater than or equal to the third threshold, then determine the standard face IR feature corresponding to the largest second matching value in the feature library, and determine the identity ID corresponding to the standard face IR feature as the user's identity ID.

[0173] Specifically, after obtaining multiple second matching values, they are sorted, and the largest second matching value is taken for operation. If the largest second matching value is greater than or equal to the third threshold, it is determined that the user has recognition authority, and the identity ID corresponding to the standard face IR feature corresponding to the largest second matching value is identified as the user's identity ID.

[0174] The third threshold is used as the threshold corresponding to the second matching set obtained by face IR feature comparison, and can be set according to the actual situation.

[0175] For example, if the third threshold is set to 0.88, and the maximum second matching value of user F8 is 0.94, and 0.94 > 0.88, then user F8 is deemed to have recognition authority. If the identity ID corresponding to the standard face IR feature corresponding to the maximum second matching value is determined to be 0168, then user F8 is deemed to be user 0168.

[0176] Step S110: If the largest second matching value among multiple second matching values ​​is less than the third threshold, then the user is determined to be a stranger.

[0177] Specifically, when the maximum second matching value obtained is less than the third threshold, it is determined that the user does not have the identification authority and the user is identified as a stranger.

[0178] For example, the maximum second matching value for user F9 is 0.75. Since 0.75 < 0.88, user F9 is deemed to lack identification authority and is identified as a stranger.

[0179] Step S201: If the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than the first limit number, then the target face RGB feature is added to the feature library as a record.

[0180] Specifically, after a user is successfully matched, the processor queries the feature library to identify the number of standard face RGB features corresponding to the identity ID. If the number is less than the first limit, the target face RGB features of the user are recorded in the feature library and bound to the corresponding identity ID. The number of standard face RGB features for that identity ID in the feature library is then incremented by 1.

[0181] The first limitation is the limit on the number of standard face RGB features corresponding to each user's identity ID in the feature library. The standard face RGB features in the feature library are collected from the same angle.

[0182] For example, assuming the second threshold is 0.9, the first limit number is 5, the maximum first matching value S1 when user F10 performs face recognition is 0.93>0.9, corresponding to the identity ID 1025 in the feature library, and the number of standard face RGB features of user 1025 in the feature library is 3<5, then the target face RGB features are added to the feature library as a record, and at this time user 1025 has 4 standard face RGB features.

[0183] Step S202: If the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to the first limit number, then obtain the face RGB quality score of the target face RGB feature and the lowest face RGB quality score among the standard face RGB features corresponding to the identity ID.

[0184] Specifically, after a user's face RGB feature is successfully matched, the processor queries the feature library to identify the number of standard face RGB features corresponding to the identity ID. If the number is equal to the first limit, the face RGB quality score of the target face RGB feature and the lowest face RGB quality score among the standard face RGB features corresponding to the identity ID are obtained. The face RGB quality score can be obtained by quantifying the face RGB features. The processor calculates the quality scores of the 128-dimensional LBP texture low-level features, the 256-dimensional HOG shape mid-level features, and the 128-dimensional depth convolution high-level features, and then sums them by weight to obtain the face RGB quality score, as shown in Formula 9.

[0185] ,

[0186] ,

[0187] ,

[0188] (9)

[0189] Among them, Q low Q represents the quality score corresponding to the bottom-level features of a 128-dimensional LBP texture. mid Q represents the quality score corresponding to the mid-layer features of the 256-dimensional HOG shape. high This represents the quality score corresponding to the high-level features of a 128-dimensional deep convolution. R represents the variance of the low-level features of a 128-dimensional LBP texture, ranging from 0 to 100. A larger value indicates more significant texture differences; the gain tends to saturate after exceeding 50. i P represents the gradient value of the i-th feature, ranging from 0 to 20. A value exceeding 10 indicates that the edge is sufficiently sharp. i This represents the probability distribution of the i-th feature. The feature entropy value represents the high-level features of deep convolution, ranging from 0 to 7. A feature entropy value of 6 corresponds to a uniform distribution of 128-dimensional high-level features, exhibiting optimal discriminative power. N low N represents the number of non-missing dimensions in the bottom-level features of a 128-dimensional LBP texture. mid N represents the number of non-missing dimensions in the mid-layer features of a 256-dimensional HOG shape. high This represents the number of non-missing dimensions in a 128-dimensional deep convolutional high-level feature.

[0190] Face RGB quality score refers to the score used by a face quality assessment model to evaluate the quality of face RGB features. It provides a unified standard for feature quality assessment and avoids low-quality features from entering the feature library and affecting matching accuracy. By comparing the quality scores of target and standard features, it can make scientific decisions on whether to replace features and ensure that high-quality and highly representative samples are always retained in the feature library.

[0191] For example, user F6 has 5 RGB facial features in the feature library, which is the first limit. The RGB quality score Q of the face is calculated. RGB The score is 95, corresponding to user 0028. In the feature library, the lowest face RGB quality score for user 0028 is 92.

[0192] Step S203: If the face RGB quality score is higher than the lowest face RGB quality score, then the target face RGB feature is used to replace the standard face RGB feature corresponding to the lowest face RGB quality score in the feature library.

[0193] Specifically, if the face RGB quality score obtained in step S202 is higher than the lowest face quality score of the identity ID in the feature library, the target face RGB features can be used to update and replace the standard face RGB features with the lowest face RGB quality score in the feature library. This method can ensure that the features in the feature library are automatically updated, and the quality of the face RGB features in the feature library becomes higher and higher, which can improve the accuracy in subsequent comparisons.

[0194] For example, the RGB quality score of user F6's face is Q. RGB Since 95 > 92, the target face RGB features of user F6 are used to replace the standard face RGB features of user 0028 in the feature library with the lowest face RGB quality score.

[0195] Step S204: If the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than the second limit number, then the target face IR feature is added to the feature library as a record.

[0196] Specifically, after a user is successfully matched, the processor queries the feature library to identify the number of standard face IR features corresponding to the identity ID. If the number is less than the second limit, the target face IR features of the user are recorded in the feature library and bound to the corresponding identity ID. The number of standard face IR features for that identity ID in the feature library is incremented by 1.

[0197] The second limitation refers to the limit on the number of standard facial IR features corresponding to each user's identity ID in the feature library. The standard facial IR features in the feature library are collected from the same angle.

[0198] For example, assuming the third threshold is 0.88, the second limit number is 8, and the maximum second matching value S2 when user F11 performs face recognition is 0.91>0.88, corresponding to the identity ID 1108 in the feature library, and the number of standard face IR features of user 1108 in the feature library is 5<8, then the target face IR feature is added to the feature library as a record, and at this time user 1108 has 6 standard face IR features.

[0199] Step S205: If the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to the second limit number, then obtain the face IR quality score of the target face IR feature and the lowest face IR quality score among the standard face IR features corresponding to the identity ID.

[0200] Specifically, after a user's face IR feature is successfully matched, the processor queries the feature library to identify the number of standard face IR features corresponding to the identity ID. If the number is equal to the second limit, the face IR quality score of the target face IR feature and the lowest face IR quality score among the standard face IR features corresponding to the identity ID are obtained. The face IR quality score is calculated by referring to Formula 10, which calculates the quality scores of the 64-dimensional absolute temperature component and the 64-dimensional thermal gradient component, and then performing a weighted summation to obtain the face IR quality score.

[0201] ,

[0202] ,

[0203] (10)

[0204] Among them, Q D Q represents the mass fraction of the 64-dimensional absolute temperature component. H N represents the mass fraction of the 64-dimensional thermal gradient components. D C represents the number of effective components of the 64-dimensional absolute temperature components, that is, the number of components whose temperature falls within the normal range for the human body. D Indicates the coverage of the absolute temperature component. N represents the gradient variance of the 64-dimensional thermal gradient components, ranging from 0 to 0.5. Gain saturates when the gradient variance approaches 0.3. H The reasonable number of components representing the 64-dimensional thermal gradient can be obtained by checking whether the range of facial temperature differences is reasonable.

[0205] Face quality IR score is a quantitative indicator used to evaluate the quality of face IR features. It is a specific numerical value (usually between 0 and 100), with higher scores indicating better face quality. By comparing the quality scores of target face IR features with those of standard face IR features, a scientific decision can be made on whether to perform feature replacement, ensuring that the feature database always retains high-quality and highly representative samples.

[0206] For example, user F12 has 8 facial IR features in the feature library, which is the second limit. The facial IR quality score Q of user F12 is calculated. IR The score is 96, corresponding to identity ID 1212 in the feature library. The minimum standard face IR quality score for user 1212 in the feature library is 90.

[0207] Step S206: If the face IR quality score is higher than the lowest face IR quality score, then the target face IR feature is used to replace the standard face IR feature corresponding to the lowest face IR quality score in the feature library.

[0208] For example, the face IR quality score Q of user F12IR Since 96 > 90, the target face IR feature of user F12 is used to replace the standard face IR feature of user 1212 with the lowest face IR quality score in the feature library.

[0209] Step S301: If the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than the first limit number, then the target face RGB features are added to the feature library as a record.

[0210] Specifically, you can refer to the embodiment described in step S201, in which the standard face RGB features in the feature library are collected from different angles. When updating features in the future, the high-quality standard face RGB features from each angle can be effectively preserved, which is beneficial to improving the accuracy of face recognition.

[0211] Step S302: If the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to the first limit number, then obtain the face RGB quality score of the target face RGB feature and the standard face RGB quality score of the standard face RGB feature corresponding to the maximum first matching value.

[0212] Specifically, after a user successfully matches the face RGB features, the processor extracts the number of standard face RGB features corresponding to the identity ID from the feature library. If the number is equal to the first limit, the face RGB quality score of the target face RGB feature and the standard face RGB quality score of the standard face RGB feature corresponding to the maximum first matching value are obtained. The face RGB quality score and the corresponding standard face RGB quality score can be obtained by referring to step S202.

[0213] For example, user F13's face RGB feature matching was successful. The number of face RGB features in the feature library is 5, which is the first limit. The face RGB quality score Q is calculated. RGB The score is 97, and the standard face RGB quality score is 94.

[0214] Step S303: If the RGB quality score of the face is higher than the RGB quality score of the standard face, then the RGB feature of the target face is used to replace the standard face RGB feature corresponding to the largest first matching value in the feature library.

[0215] Specifically, if the RGB quality score of the face obtained in step S302 is higher than the corresponding standard face quality score in the feature library, the target face RGB features can be used to update and replace the corresponding standard face RGB features in the feature library. This method can ensure that the features in the feature library can be automatically updated. Since each feature has an angle distinction, the quality of the face RGB features at each angle in the feature library is getting higher and higher, and the accuracy in subsequent comparisons can also be improved.

[0216] For example, the RGB quality score of user F13's face is Q. RGB If 97 > 94, then the RGB features of the target face obtained in this identification can replace the standard face RGB features corresponding to the largest first matching value in the feature library.

[0217] Step S304: If the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than the second limit number, then the target face IR feature is added to the feature library as a record.

[0218] Specifically, you can refer to the embodiment described in step S204, in which the standard face IR features in the feature library are collected from different angles. When updating features in the future, the high-quality standard face IR features from each angle can be effectively preserved, which is beneficial to improving the accuracy of face recognition.

[0219] Step S305: If the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to the second limit number, then obtain the face IR quality score of the target face IR feature and the standard face IR quality score of the standard face IR feature corresponding to the maximum second matching value.

[0220] Specifically, after a user compares facial IR features, if the number of standard facial IR features corresponding to the identity ID extracted from the feature library is equal to the second limit number, then the facial IR quality score of the target facial IR feature and the standard facial IR quality score of the standard facial IR feature corresponding to the maximum first matching value are obtained. The facial IR quality score and the corresponding standard facial IR quality score can be calculated by referring to step S205.

[0221] For example, user F14 passed the face IR feature comparison, and the corresponding identity ID in the feature library is 1425. The number of standard face IR features for user 1425 is 8, which is equal to the second limit. Then, the face IR quality score of F14 and the standard face IR quality score of the standard face IR features corresponding to the successful comparison are obtained. The face IR quality score of F14 is 98, and the corresponding standard face IR quality score is 96.

[0222] Step S306: If the face IR quality score is higher than the standard face IR quality score, then the standard face IR feature corresponding to the second largest matching value in the feature library is replaced with the target face IR feature.

[0223] Specifically, if the face IR quality score obtained in step S304 is higher than the corresponding standard face quality score in the feature library, the target face IR feature can be used to update and replace the corresponding standard face IR feature in the feature library. This method can ensure that the features in the feature library can be automatically updated. Since each feature has an angle distinction, the quality of face IR features at each angle in the feature library becomes higher and higher, and the accuracy in subsequent comparisons can also be improved.

[0224] For example, the face IR quality score Q of user F14 IR If 98 > 96, then the target face IR feature obtained in this identification can replace the standard face IR feature corresponding to the largest first matching value in the feature library.

[0225] Reference Figure 2 This application provides a face recognition device, which includes an image acquisition module, a liveness verification module, a feature extraction module, and a feature comparison module.

[0226] The image acquisition module is used to acquire the user's RGB image and IR image of the face;

[0227] The liveness verification module is used to perform binocular liveness verification on users based on RGB and IR images of their faces.

[0228] The feature extraction module is used to obtain the ambient light intensity after the binocular liveness verification is passed; and to identify the target face RGB features corresponding to the face RGB image and the target face IR features corresponding to the face IR image respectively.

[0229] The feature comparison module is used to: If the illumination intensity is greater than or equal to a first threshold, match the target face RGB features with standard face RGB features in the feature library to obtain multiple first matching values; if the largest first matching value among the multiple first matching values ​​is greater than or equal to a second threshold, determine the standard face RGB feature corresponding to the largest first matching value in the feature library, and determine the identity ID corresponding to the standard face RGB feature as the user's identity ID; if the largest first matching value among the multiple first matching values ​​is less than the second threshold, determine that the user is a stranger; if the illumination intensity is less than the first threshold, match the target face IR features with standard face IR features in the feature library to obtain multiple second matching values; if the largest second matching value among the multiple second matching values ​​is greater than or equal to a third threshold, determine the standard face IR feature corresponding to the largest second matching value in the feature library, and determine the identity ID corresponding to the standard face IR feature as the user's identity ID; if the largest second matching value among the multiple second matching values ​​is less than the third threshold, determine that the user is a stranger.

[0230] Based on the above embodiments, the liveness verification module is further used to perform face detection on the face RGB image and face IR image. If no face is detected in the face RGB image and / or face IR image, the binocular liveness verification is deemed to have failed. If a face is detected in the face RGB image and face IR image, contrast detection, blur detection, and skin color region integrity detection are performed on the face in the face RGB image to obtain a first detection result, and infrared radiation intensity distribution detection is performed on the face IR image to obtain a second detection result. If the first detection result and / or the second detection result fail, the binocular liveness verification is deemed to have failed. If both the first and second detection results pass, key points of the face in the face RGB image and face IR image are extracted, and prompt instructions are output to prompt the user to complete dynamic actions. Based on the dynamic actions, cross-frame tracking of key points is performed to obtain motion trajectories. Based on the motion trajectories, the continuity of the dynamic actions is judged. If the continuity is unqualified, the binocular liveness verification is deemed to have failed. If the continuity is qualified, the binocular liveness verification is deemed to have passed.

[0231] Based on the above embodiments, the feature extraction module is also used to remove noise from the face RGB image through bilateral filtering, and to obtain a face RGB texture enhancement image by using histogram equalization to denoise the face RGB image. The ROI region containing the complete face is then extracted from the face RGB texture enhancement image. The ROI region is identified to obtain the LBP texture low-level features, HOG shape mid-level features, and deep convolution high-level features as target face RGB features. The thermal noise points of the face IR image are removed through morphological filtering, and the pixel values ​​of the face IR image with thermal noise points removed are mapped to a fixed range by temperature normalization to highlight the facial temperature difference of the face IR image, thus obtaining a face facial temperature difference image. The temperature distribution features and thermal gradient features of the face facial temperature difference image are identified as target face IR features.

[0232] Based on the above embodiments, the feature comparison module is further used to obtain a 512-dimensional target feature vector corresponding to the RGB features of the target face. The 512-dimensional target feature vector includes a 128-dimensional target LBP texture bottom-level feature vector, a 256-dimensional target HOG shape mid-level feature vector, and a 128-dimensional target depth convolution high-level feature vector. It also obtains a 512-dimensional standard feature vector corresponding to the RGB features of the standard face. This standard feature vector includes a 128-dimensional standard LBP texture bottom-level feature vector, a 256-dimensional standard HOG shape mid-level feature vector, and a 128-dimensional standard depth convolution high-level feature vector. The 512-dimensional target feature vector and the 512-dimensional standard feature vector are multiplied according to their respective dimensions to obtain the feature matching values ​​corresponding to each dimension of the target face RGB features. The 128-dimensional target LBP texture bottom-level feature vector and the 256-dimensional target HOG shape mid-level feature vector are calculated respectively. The arithmetic mean of the feature matching values ​​of each dimension corresponding to the layer features and the high-level features of the 128-dimensional target depth convolution is used to obtain the bottom-layer feature matching value, the middle-layer feature matching value, and the high-level feature matching value. If the illumination intensity is greater than or equal to the first threshold and less than the fourth threshold, the bottom-layer feature matching value, the middle-layer feature matching value, and the high-level feature matching value are assigned a first weight, and the fourth threshold is greater than the first threshold. Based on the first weight, the bottom-layer feature matching value, the middle-layer feature matching value, and the high-level feature matching value are weighted and summed to obtain the first matching value. If the illumination intensity is greater than or equal to the fourth threshold, the weight ratio of the high-level feature matching value is increased, and the bottom-layer feature matching value, the middle-layer feature matching value, and the high-level feature matching value are redistributed a second weight. Based on the second weight, the bottom-layer feature matching value, the middle-layer feature matching value, and the high-level feature matching value are weighted and summed to obtain the first matching value.

[0233] Based on the above embodiments, the feature comparison module is further used to obtain the target absolute temperature component of the target face IR feature and the standard absolute temperature component of the standard face IR feature; to obtain the Euclidean distance between the target face IR feature and the standard face IR feature by calculating the target absolute temperature component and the standard absolute temperature component; to perform normalization calculation on the Euclidean distance to obtain the absolute temperature matching value of the target face IR feature; to obtain the target thermal gradient component of the target face IR feature and the standard thermal gradient component of the standard face IR feature; to calculate the cosine similarity between the target thermal gradient component and the standard thermal gradient component as the thermal gradient matching value of the target face IR feature; to assign a third weight to the absolute temperature matching value and the thermal gradient matching value; and to perform a weighted summation of the absolute temperature matching value and the thermal gradient matching value based on the third weight to obtain a second matching value.

[0234] Based on the above embodiments, the face recognition device further includes:

[0235] The feature library update module is used to add the target face RGB feature as a record to the feature library if the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than the first limit number; if the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to the first limit number, then the module obtains the face RGB quality score of the target face RGB feature and the lowest face RGB quality score among the standard face RGB features corresponding to the identity ID; if the face RGB quality score is higher than the lowest face RGB quality score, then the module replaces the target face RGB feature corresponding to the lowest face RGB quality score in the feature library. The standard face RGB features are used; if the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than the second limit, then the target face IR feature is added to the feature library as a record; if the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to the second limit, then the face IR quality score of the target face IR feature and the lowest face IR quality score among the standard face IR features corresponding to the identity ID are obtained; if the face IR quality score is higher than the lowest face IR quality score, then the standard face IR feature corresponding to the lowest face IR quality score in the feature library is replaced by the target face IR feature.

[0236] Based on the above embodiments, the standard face RGB features and standard face IR features of each identity ID in the feature library are extracted from face RGB images and face IR images from different angles. The face recognition device also includes:

[0237] The feature library update module is used to add the target face RGB feature as a record to the feature library if the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than the first limit number; if the maximum first matching value is greater than or equal to the second threshold, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to the first limit number, then the module obtains the face RGB quality score of the target face RGB feature and the standard face RGB quality score of the standard face RGB feature corresponding to the maximum first matching value; if the face RGB quality score is higher than the standard face RGB quality score, then the module replaces the record corresponding to the maximum first matching value in the feature library with the target face RGB feature. The standard face RGB features are used; if the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than the second limit, then the target face IR feature is added to the feature library as a record; if the maximum second matching value is greater than or equal to the third threshold, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to the second limit, then the face IR quality score of the target face IR feature and the standard face IR quality score of the standard face IR feature corresponding to the maximum second matching value are obtained; if the face IR quality score is higher than the standard face IR quality score, then the standard face IR feature corresponding to the maximum second matching value in the feature library is replaced by the target face IR feature.

[0238] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0239] This application provides an electronic device, such as... Figure 3 As shown, the electronic device 400 may include: at least one processor 401, at least one network interface 404, user interface 403, memory 405, and at least one communication bus 402.

[0240] The communication bus 402 is used to enable communication between these components.

[0241] The user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0242] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0243] The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server 400 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, and by calling data stored in the memory 405. Optionally, the processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 401 and may be implemented as a separate chip.

[0244] The memory 405 may include random access memory (RAM) or read-only memory. Optionally, the memory 405 may include a non-transitory computer-readable storage medium. The memory 405 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 405 may also be at least one storage device located remotely from the aforementioned processor 401. Figure 3 As shown, the memory 405, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a face recognition method.

[0245] exist Figure 3 In the electronic device 400 shown, the user interface 403 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 401 can be used to call an application program storing a face recognition method in the memory 405. When executed by one or more processors, the electronic device executes the face recognition method described in the embodiments of this application.

[0246] This application provides a computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause the face recognition method provided in this application to be performed.

[0247] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A face recognition method, characterized by, The method comprises the following steps: obtaining a face RGB image and a face IR image of a user; performing binocular liveness verification on the user based on the face RGB image and the face IR image; after the binocular liveness verification is passed, obtaining the light intensity of a current environment; removing noise in the face RGB image through bilateral filtering, performing global histogram equalization on the face RGB image after noise removal, obtaining a face RGB texture enhanced image, and intercepting an ROI region of the face RGB texture enhanced image containing a complete face; identifying the ROI region to obtain a 128-dimensional LBP texture bottom layer feature, a 256-dimensional HOG shape middle layer feature, and a 128-dimensional deep convolution high layer feature as a target face RGB feature; removing thermal noise points in the face IR image through morphological filtering, and mapping pixel values of the face IR image after removal of the thermal noise points to a fixed range through temperature normalization to highlight the face temperature difference of the face IR image, thereby obtaining a face temperature difference image; identifying the temperature distribution feature and the thermal gradient feature of the face temperature difference image as a target face IR feature; if the light intensity is greater than or equal to a first threshold value, then matching the target face RGB feature with each standard face RGB feature in a feature library to obtain a plurality of first matching values; if a maximum first matching value in the plurality of first matching values is greater than or equal to a second threshold value, then determining a standard face RGB feature corresponding to the maximum first matching value in the feature library, and determining an identity ID corresponding to the standard face RGB feature as an identity ID of the user; if the maximum first matching value in the plurality of first matching values is less than the second threshold value, then determining that the user is a stranger; if the light intensity is less than the first threshold value, then matching the target face IR feature with each standard face IR feature in a feature library to obtain a plurality of second matching values; if a maximum second matching value in the plurality of second matching values is greater than or equal to a third threshold value, then determining a standard face IR feature corresponding to the maximum second matching value in the feature library, and determining an identity ID corresponding to the standard face IR feature as an identity ID of the user; if the maximum second matching value in the plurality of second matching values is less than the third threshold value, then determining that the user is a stranger; wherein the matching of the target face RGB feature with each face RGB feature in the feature library to obtain a plurality of first matching values comprises: obtaining a 512-dimensional target feature vector corresponding to the target face RGB feature, the 512-dimensional target feature vector comprising a 128-dimensional target LBP texture bottom layer feature vector, a 256-dimensional target HOG shape middle layer feature vector, and a 128-dimensional target deep convolution high layer feature vector; obtaining a 512-dimensional standard feature vector corresponding to the standard face RGB feature, the 512-dimensional standard feature vector comprising a 128-dimensional standard LBP texture bottom layer feature vector, a 256-dimensional standard HOG shape middle layer feature vector, and a 128-dimensional standard deep convolution high layer feature vector; multiply the 512-dimensional target feature vector with the 512-dimensional standard feature vector corresponding to each dimension to obtain a feature matching value corresponding to each dimension of the target face RGB feature; respectively calculate the arithmetic mean of the feature matching values corresponding to the 128-dimensional target LBP texture bottom layer feature, the 256-dimensional target HOG shape middle layer feature and the 128-dimensional target depth convolution high layer feature to obtain a bottom layer feature matching value, a middle layer feature matching value and a high layer feature matching value; if the illumination intensity is greater than or equal to the first threshold value and less than a fourth threshold value, assign a first weight to the bottom layer feature matching value, the middle layer feature matching value and the high layer feature matching value, and the fourth threshold value is greater than the first threshold value; based on the first weight, perform weighted summation on the bottom layer feature matching value, the middle layer feature matching value and the high layer feature matching value to obtain the first matching value; if the illumination intensity is greater than or equal to the fourth threshold value, increase the weight proportion of the high layer feature matching value, and reassign a second weight to the bottom layer feature matching value, the middle layer feature matching value and the high layer feature matching value; based on the second weight, perform weighted summation on the bottom layer feature matching value, the middle layer feature matching value and the high layer feature matching value to obtain the first matching value.

2. The method of claim 1, wherein, The binocular living body verification of the user based on the face RGB image and the face IR image includes: performing face detection on the face RGB image and the face IR image, if no face is detected in the face RGB image and / or the face IR image, determining that the binocular living body verification fails; if a face is detected in the face RGB image and the face IR image, performing contrast detection, blur detection and skin color region integrity detection on the face in the face RGB image to obtain a first detection result, and performing infrared radiation intensity distribution detection on the face IR image to obtain a second detection result; if the first detection result and / or the second detection result fails, it is determined that the binocular living body verification fails; if the first detection result and the second detection result both pass, extracting the key points of the face in the face RGB image and the face IR image, and outputting a prompt instruction, the prompt instruction is used to prompt the user to complete a dynamic action; based on the dynamic action, cross-frame tracking is performed on the key points to obtain a motion trajectory; based on the motion trajectory, the continuity of the dynamic action is judged; if the continuity is unqualified, it is determined that the binocular living body verification fails; if the continuity is qualified, it is determined that the binocular living body verification passes.

3. The method of claim 1, wherein, The target face IR feature is matched with each face IR feature in the feature library to obtain a plurality of second matching values, including: obtaining a target absolute temperature component of the target face IR feature and a standard absolute temperature component of the standard face IR feature; obtaining an Euclidean distance between the target face IR feature and the standard face IR feature by calculating the target absolute temperature component and the standard absolute temperature component; performing a normalization calculation on the Euclidean distance to obtain an absolute temperature matching value of the target face IR feature; obtaining a target thermal gradient component of the target face IR feature and a standard thermal gradient component of the standard face IR feature; calculating a cosine similarity between the target thermal gradient component and the standard thermal gradient component as a thermal gradient matching value of the target face IR feature; distributing a third weight to the absolute temperature matching value and the thermal gradient matching value; performing a weighted summation on the absolute temperature matching value and the thermal gradient matching value based on the third weight to obtain the second matching value.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: if the maximum first matching value is greater than or equal to a second threshold value, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than a first limit number, adding the target face RGB feature as a record to the feature library; if the maximum first matching value is greater than or equal to the second threshold value, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to the first limit number, obtaining a face RGB quality score of the target face RGB feature and a lowest face RGB quality score in the standard face RGB features corresponding to the identity ID; if the face RGB quality score is higher than the lowest face RGB quality score, replacing the standard face RGB feature corresponding to the lowest face RGB quality score in the feature library with the target face RGB feature; if the maximum second matching value is greater than or equal to a third threshold value, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than a second limit number, adding the target face IR feature as a record to the feature library; if the maximum second matching value is greater than or equal to the third threshold value, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to the second limit number, obtaining a face IR quality score of the target face IR feature and a lowest face IR quality score in the standard face IR features corresponding to the identity ID; if the face IR quality score is higher than the lowest face IR quality score, replacing the standard face IR feature corresponding to the lowest face IR quality score in the feature library with the target face IR feature.

5. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: if the maximum first matching value is greater than or equal to a second threshold value, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is less than a first limit number, adding the target face RGB feature as a record to the feature library; If the maximum first matching value is greater than or equal to a second threshold value, and the number of standard face RGB features corresponding to the identity ID recorded in the feature library is equal to a first limit number, then a face RGB quality score of the target face RGB feature and a standard face RGB quality score of the standard face RGB feature corresponding to the maximum first matching value are obtained; If the face RGB quality score is higher than the standard face RGB quality score, then the target face RGB feature is used to replace the standard face RGB feature corresponding to the maximum first matching value in the feature library; If the maximum second matching value is greater than or equal to a third threshold value, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is less than a second limit number, then the target face IR feature is added to the feature library as a record; If the maximum second matching value is greater than or equal to a third threshold value, and the number of standard face IR features corresponding to the identity ID recorded in the feature library is equal to a second limit number, then a face IR quality score of the target face IR feature and a standard face IR quality score of the standard face IR feature corresponding to the maximum second matching value are obtained; If the face IR quality score is higher than the standard face IR quality score, then the target face IR feature is used to replace the standard face IR feature corresponding to the maximum second matching value in the feature library.

6. A face recognition apparatus, characterized by comprising: The method comprises: an image acquisition module, configured to acquire a face RGB image and a face IR image of a user; a living body verification module, configured to perform binocular living body verification on the user based on the face RGB image and the face IR image; a feature extraction module, configured to, after the binocular living body verification passes, acquire a light intensity of a current environment; remove noise in the face RGB image through bilateral filtering, perform global histogram equalization on the face RGB image after noise removal, to obtain a face RGB texture enhanced image, and cut an ROI region of the face RGB texture enhanced image containing a complete face; identify the ROI region to obtain a 128-dimensional LBP texture bottom feature, a 256-dimensional HOG shape middle feature, and a 128-dimensional deep convolution high feature, as a target face RGB feature; remove thermal noise points of the face IR image through morphological filtering, map pixel values of the face IR image after removal of the thermal noise points to a fixed range through temperature normalization, highlight face temperature differences of the face IR image, to obtain a face temperature difference image; identify temperature distribution features and thermal gradient features of the face temperature difference image, as a target face IR feature; and a feature library, configured to record the target face RGB feature and the target face IR feature. The feature comparison module is configured to: if the illumination intensity is greater than or equal to a first threshold value, match the target face RGB feature with each standard face RGB feature in a feature library to obtain a plurality of first matching values; if a maximum first matching value in the plurality of first matching values is greater than or equal to a second threshold value, determine a standard face RGB feature corresponding to the maximum first matching value in the feature library, and determine an identity ID corresponding to the standard face RGB feature as an identity ID of the user; if the maximum first matching value in the plurality of first matching values is less than the second threshold value, determine that the user is a stranger; if the illumination intensity is less than the first threshold value, match the target face IR feature with each standard face IR feature in a feature library to obtain a plurality of second matching values; if a maximum second matching value in the plurality of second matching values is greater than or equal to a third threshold value, determine a standard face IR feature corresponding to the maximum second matching value in the feature library, and determine an identity ID corresponding to the standard face IR feature as an identity ID of the user; if the maximum second matching value in the plurality of second matching values is less than the third threshold value, determine that the user is a stranger. The feature comparison module is further configured to obtain a 512-dimensional target feature vector corresponding to the target face RGB feature, the 512-dimensional target feature vector comprising a 128-dimensional target LBP texture bottom layer feature vector, a 256-dimensional target HOG shape middle layer feature vector, and a 128-dimensional target deep convolution high layer feature vector; obtain a 512-dimensional standard feature vector corresponding to the standard face RGB feature, the 512-dimensional standard feature vector comprising a 128-dimensional standard LBP texture bottom layer feature vector, a 256-dimensional standard HOG shape middle layer feature vector, and a 128-dimensional standard deep convolution high layer feature vector; multiply the 512-dimensional target feature vector and the 512-dimensional standard feature vector in each dimension to obtain a feature matching value corresponding to each dimension of the target face RGB feature; calculate an arithmetic mean of the feature matching values corresponding to the 128-dimensional target LBP texture bottom layer feature, the 256-dimensional target HOG shape middle layer feature, and the 128-dimensional target deep convolution high layer feature, respectively, to obtain a bottom layer feature matching value, a middle layer feature matching value, and a high layer feature matching value; if the illumination intensity is greater than or equal to the first threshold value and less than a fourth threshold value, assign a first weight to the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value, the fourth threshold value being greater than the first threshold value; based on the first weight, perform weighted summation on the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value to obtain the first matching value; if the illumination intensity is greater than or equal to the fourth threshold value, increase the weight proportion of the high layer feature matching value, and reassign a second weight to the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value; based on the second weight, perform weighted summation on the bottom layer feature matching value, the middle layer feature matching value, and the high layer feature matching value to obtain the first matching value.

7. An electronic device, comprising: The electronic device comprises a memory, a processor, a user interface, and a network interface, the memory being configured to store instructions, the user interface and the network interface being configured to communicate with other devices, and the processor being configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed, perform the method of any one of claims 1 to 5.

Images