Face recognition method based on point cloud data and RGB image sequential verification

Abstract

The invention provides a face recognition method based on point cloud data and RGB image sequential verification, and the method comprises the following steps: collecting two types of data at the same time, completing the face detection and cutting of an RGB image, combining histogram equalization and illumination compensation, extracting unitized features, comparing the unitized features with a feature library, and carrying out the confidence calibration. And according to the first upper threshold value and the first lower threshold value, primary judgment of passing, re-verification or refusal is made. After the RGB image passes verification, selecting a point cloud corresponding to a human face region, executing voxel downsampling, counting abnormal point removal and normal estimation, calculating local geometric description, performing robust registration, and constructing a single geometric similarity index based on a median absolute deviation of a point-to-surface residual error and quantile statistics; and outputting passing, re-checking or refusing according to the second upper threshold value and the second lower threshold value. Only when the two stages pass, an identity conclusion and confidence information are given; and when any stage is not passed or re-verification is not passed, outputting a rejection result or triggering a warning or auxiliary verification.

Description

A face recognition method based on sequential verification of point cloud data and RGB images Technical Field

[0001] This invention relates to computer vision, measurement and control technology and instruments, and specifically to a face recognition method based on sequential verification of point cloud data and RGB images. Background Technology

[0002] With the widespread deployment of applications such as intelligent urban surveillance, access control, and identity authentication, the demand for real-time, accurate, and robust facial recognition capabilities is increasing. Compared with traditional manual inspections, online facial recognition systems can significantly reduce labor costs, improve response speed, and enhance security, thus experiencing rapid development in fields such as security, finance, transportation, and smart homes. In practical applications, different shooting environments (such as low light, high brightness, and strong backlight) and the inspected target (such as occlusion, side profile, and facial expression changes) pose significant challenges to recognition accuracy; in addition, camera angle, resolution, and passive anti-spoofing scenarios also affect recognition reliability.

[0003] Existing single-modal face recognition methods each have their limitations: RGB image-based appearance recognition methods (such as those based on LBPH, traditional features, or deep models based on convolutional neural networks) perform well when texture and color information is sufficient, but are prone to misjudgments under varying lighting conditions, makeup, masks, or planar spoofing attacks (photo or video replay); while depth- or point cloud-based geometric recognition methods utilize 3D shape information to combat lighting and appearance camouflage, but point cloud data may lack sufficient information when acquisition resolution, occlusion, and noise levels are high, and directly applying 3D registration or descriptor matching is computationally intensive, which is not conducive to real-time deployment with limited resources. Therefore, single-modal methods cannot simultaneously achieve accuracy, robustness, and efficiency.

[0004] To compensate for the shortcomings of single-modality methods, existing technologies propose multimodal fusion strategies that improve recognition robustness by simultaneously utilizing two-dimensional appearance and three-dimensional geometric information. However, typical fusion methods usually revolve around parallel fusion at the feature level or decision level. On the one hand, this significantly increases computational and storage overhead, which is not conducive to deployment on edge devices or real-time systems. On the other hand, during fusion, the two modalities are required to be strictly aligned in space and time, which is susceptible to synchronization errors and viewpoint differences in practice. Thirdly, the trade-offs of fusion strategies in different scenarios are complex, making it difficult to control the false alarm rate and false negative rate while maintaining high accuracy.

[0005] The quality of point cloud and RGB data acquisition is significantly affected by sensor characteristics and the environment. Common depth cameras perform unpredictably in strong outdoor light, at long distances, or on highly reflective surfaces, often resulting in point clouds containing holes, redundancy, and outliers. RGB images, on the other hand, suffer severe detail loss under low light or strong backlighting. Faced with these real-world challenges, designing a recognition process that balances efficiency and robustness, and achieving reliable sequential verification under hardware depth camera standards, has become an urgent engineering problem to be solved. Summary of the Invention

[0006] To address the aforementioned shortcomings, this invention provides a face recognition method based on sequential verification using point cloud data and RGB images. It proposes a sequential dual-mode verification strategy: first, rapid appearance verification based on RGB images; then, if the RGB verification passes, precise geometric verification of the corresponding point cloud. This strategy significantly reduces the frequency of point cloud processing and the overall system computational cost while maintaining recognition accuracy, thus possessing significant engineering value in real-time monitoring and embedded deployment scenarios. This strategy should also be complemented by lightweight pre-screening, point cloud denoising and downsampling, registration and descriptor optimization, and compensation techniques for illumination and sensor noise to improve the system's stability and applicability in complex environments.

[0007] This invention provides the following technical solution: a face recognition method based on sequential verification of point cloud data and RGB images, comprising the following steps:

[0008] Acquire RGB image data and point cloud data at the same time point;

[0009] The RGB image data is subjected to face detection, cropping and feature extraction to obtain RGB face features, which are then compared with a face feature database to obtain RGB verification similarity, and thus the RGB image verification result is obtained.

[0010] Based on the results of RGB image verification, the user is selected to proceed to point cloud geometric verification or RGB image re-verification. When proceeding to point cloud geometric verification, geometric feature extraction and matching operations are performed on the point cloud data corresponding to the RGB face region to obtain the point cloud geometric verification similarity, and thus obtain the result of point cloud geometric verification.

[0011] Based on the results of point cloud geometric verification, make a face recognition identity determination result or perform point cloud geometric re-verification;

[0012] The face recognition passes only if the point cloud geometry verification meets the passing conditions. At this time, the identity result of the identified object and the corresponding confidence information are output; otherwise, the face recognition fails or is uncertain.

[0013] When the RGB image re-verification or point cloud geometry re-verification fails, the current recognition result is submitted to the warning and fallback verification system to trigger at least one of the following methods: password verification, fingerprint verification, or manual review, to complete the identity verification.

[0014] Furthermore, the RGB image verification employs a hierarchical judgment mechanism, including:

[0015] When the RGB verification similarity is higher than the first upper threshold, point cloud geometric verification is triggered directly;

[0016] When the RGB verification similarity is lower than the first lower threshold, the RGB verification is deemed to have failed and the current recognition process is terminated.

[0017] When the RGB verification similarity is between the first upper threshold and the first lower threshold, it is marked as a blurred state and enters the RGB re-verification process;

[0018] The RGB re-verification process includes:

[0019] For the same target, RGB image data are reacquired at different time frames or under different sampling conditions, and the RGB image verification is performed again to calculate the RGB verification similarity.

[0020] Statistical fusion of multiple RGB verification similarity results;

[0021] When the fusion result is higher than the first upper threshold and each result is not lower than the first lower threshold, the point cloud geometry verification stage is allowed; otherwise, an alarm is triggered or the auxiliary identity verification process is initiated.

[0022] Furthermore, the point cloud geometric verification also employs a hierarchical judgment mechanism, including:

[0023] When the geometric similarity of the point cloud is higher than the second upper threshold, the point cloud verification is deemed to have passed.

[0024] When the geometric similarity of the point cloud is lower than the second lower threshold, the point cloud verification is deemed to have failed.

[0025] When the geometric similarity of the point cloud is between the second upper threshold and the second lower threshold, the point cloud re-verification process begins.

[0026] The point cloud re-verification process includes:

[0027] Reacquire point cloud data of the target area or extract face point cloud set from multiple consecutive frames of point cloud data and perform point cloud geometry verification again;

[0028] Calculate the geometric similarity of the point cloud for each point cloud set, and perform set-level fusion judgment;

[0029] When the geometric similarity of the set-level point cloud is higher than the second upper threshold within a specified number of times, the point cloud verification is deemed to be passed.

[0030] When the geometric similarity of the set-level point cloud is lower than the second lower threshold within a specified number of times, an alarm is triggered or the auxiliary identity verification process is entered or a rejection result is output.

[0031] In other cases, the result is determined to be uncertain and point cloud re-verification continues, or an alarm is triggered or an auxiliary identity verification process is entered after the number of attempts is exhausted.

[0032] Furthermore, the RGB image verification step includes:

[0033] Face detection is performed on the input RGB face image to obtain the face bounding box. Based on the face bounding box, the face region is cropped and scaled to a preset size. Image equalization and illumination compensation processing are then performed on the cropped image to obtain the preprocessed face image.

[0034] Based on the preprocessed face image, a face descriptor is extracted. The face descriptor extraction method is either the LBPH method or the CNN method. The extracted face descriptor is then normalized to obtain a unit feature vector.

[0035] The similarity between the unit feature vector and the reference feature vector in the face feature library is calculated, and the confidence level of the similarity is calibrated to obtain the RGB verification similarity. Based on the comparison results of the RGB verification similarity with the first upper threshold and the first lower threshold, the RGB image verification result is output and the subsequent sequential verification process is triggered.

[0036] Furthermore, the step of obtaining the preprocessed face image includes:

[0037] For each input RGB face image, a face detector is used to detect the face border, and the face region is cropped and scaled to a fixed size image with a preset side length based on the face border.

[0038] Image equalization processing is performed on the cropped and scaled face image to obtain an equalized image. Specifically, when extracting face descriptors using LBPH, the cropped and scaled image is first converted to grayscale before histogram equalization or contrast-limited adaptive histogram equalization processing is performed.

[0039] The equalized image is subjected to illumination compensation to obtain an illumination-compensated image. The illumination compensation includes nonlinear correction of pixel intensity based on gamma coefficients. When the contrast adaptive histogram equalization process is limited and a preset enhancement intensity is reached, the illumination compensation step can be omitted to avoid over-enhancement.

[0040] Furthermore, the step of extracting face descriptors and obtaining unit feature vectors based on the preprocessed face image, when using the LBPH method, includes:

[0041] Local binary pattern (LBP) coding is calculated pixel-by-pixel for the preprocessed face image using a preset radius and a preset number of neighborhood samples.

[0042] The face image is divided into a preset number of sub-grids, and the LBP encoded histogram of each sub-grid is statistically analyzed and normalized.

[0043] The normalized histograms of each subgrid are concatenated in sequence to obtain the LBPH feature vector, and the LBPH feature vector is subjected to L2 normalization to obtain the unit feature vector.

[0044] When using the CNN method, the following are included:

[0045] Pixel normalization is performed on the preprocessed face image. The pixels are normalized based on the channel mean and channel standard deviation of the training set to obtain the network input tensor.

[0046] The network input tensor is input into the trained convolutional neural network for forward inference to obtain the embedded feature vector, and the embedded feature vector is subjected to L2 normalization to obtain the unit feature vector.

[0047] The unit feature vector serves as the input for subsequent similarity calculations, enabling different descriptor extraction methods to be compared uniformly under the same threshold judgment framework.

[0048] Furthermore, the point cloud geometric verification includes:

[0049] Preprocessing is performed on the source point cloud and the reference point cloud, including voxel downsampling, outlier removal, and normal estimation, so that the two sets of point clouds meet the stability requirements of subsequent matching and consistency evaluation in terms of point density, noise, and local geometric representation.

[0050] The preprocessed point cloud is used to generate a local geometric descriptor and establish a point-to-point correspondence. An iterative robust estimation method is used to select the set of interior points that meet the error constraints from the candidate correspondences. The optimal rigid body transformation parameters are obtained by minimizing the registration residual based on the set of interior points, so as to achieve robust registration from the source point cloud to the reference point cloud.

[0051] The normal orthogonal residual is calculated based on the optimal rigid body transformation parameters, and the residual is robustly normalized and statistically aggregated to obtain the point cloud geometric verification similarity. The point cloud geometric verification similarity is compared with the second upper threshold and the second lower threshold, and the result of verification, review or rejection is output.

[0052] Furthermore, the preprocessing steps for the source point cloud and the reference point cloud include:

[0053] Voxel grid downsampling is performed on the source point cloud and the reference point cloud using a preset voxel side length. Points falling within the same voxel are replaced by their centroid points, thereby obtaining the voxel centroid set after downsampling.

[0054] For the point cloud after voxel grid downsampling, the average Euclidean distance from each point to its neighboring points is calculated based on the number of neighboring points in the neighborhood of each point. Points whose average Euclidean distance exceeds the threshold determined by the global mean and standard deviation are identified as outliers and removed.

[0055] For the point cloud after removing outliers, a local three-dimensional spherical neighborhood is constructed with a preset neighborhood radius or a preset number of neighborhood points. A covariance matrix is ​​formed based on the neighborhood point set and eigenvalue decomposition is performed. The eigenvector corresponding to the smallest eigenvalue is taken as the estimated normal vector of the point.

[0056] Furthermore, the steps for identifying and removing outliers include:

[0057] After preprocessing, local descriptors of the source point cloud and the reference point cloud are processed in a preset statistical neighborhood, and the local descriptors are normalized to obtain local feature vectors.

[0058] Based on the local feature vector, the corresponding point with the smallest descriptor distance for each source point in the source point cloud is determined in the reference point cloud, and a candidate matching pair set between the source point and the corresponding point is established.

[0059] Robust consistency screening is performed on the candidate matching pair set to retain inlier matching pairs that satisfy the local description pair retention threshold constraint. Based on the inlier matching pair set, a residual minimization model is constructed and the optimal rigid body transformation parameters from the source point cloud to the reference point cloud are obtained by solving the model.

[0060] Furthermore, the step of calculating the geometric verification similarity of the point cloud includes: based on the obtained optimal rigid body transformation parameters, performing a rigid body transformation on the source point in the source point cloud and determining its corresponding point in the reference point cloud, and using the estimated normal vector of the corresponding point to calculate the normal orthogonal residual of the transformed source point relative to the local tangent plane of the corresponding reference point.

[0061] Robust scaling estimation and scaling normalization are performed on the normal orthogonal residuals, and upper quantile statistics are extracted from the normalized residual set to suppress the influence of heavy-tailed distribution and extreme outliers on the statistical results.

[0062] The upper quantile statistics are constructed into a point cloud geometric verification similarity index through monotonically nonlinear mapping. The point cloud geometric verification similarity index is compared with the second upper threshold and the second lower threshold to determine whether the point cloud geometric verification is passed, enters re-verification, or directly outputs a rejection result.

[0063] The beneficial effects of this invention are as follows:

[0064] 1. The technical solution of this application first reduces the impact of device differences and complex lighting on face images through grayscale conversion, histogram equalization, and appropriate illumination compensation, so that key textures are reliably presented in both dark and bright areas. Subsequently, local binary mode histograms and convolutional neural networks are used to extract representations, and the output vectors are normalized to make judgments with a unified similarity and confidence scale, thereby maintaining a consistent threshold system and stable on-site performance across different feature types.

[0065] 2. The technical approach of this application first performs rapid image-based judgment in its process design, and only performs point cloud geometric verification on samples that meet the entry conditions; re-verification is introduced for boundary samples to suppress fluctuations caused by occasional jitter and occlusion through multi-frame consistency. In the point cloud stage, voxel downsampling, statistical outlier removal, and normal estimation are adopted, and a single geometric similarity index is constructed using median absolute deviation and quantiles to reduce the impact of outliers and density variations on the results, making the threshold easier to solidify over the long term and the judgment more reproducible.

[0066] 3. The technical solution of this application compares the result of RGB image verification with a first upper threshold and a second lower threshold. Only the original sampled image that passes RGB image verification is then subjected to point cloud geometric verification. Only when both verifications pass is the result of successful face recognition output. This is a common sequential verification strategy, which significantly reduces the call frequency of point cloud registration, thereby reducing the average computational load and significantly improving the system response speed. Dual-mode complementarity enhances recognition robustness—the RGB image verification step can quickly and efficiently screen candidates and resist the stability of appearance details, while the point cloud module provides geometric verification of occlusion, makeup, and planar disguise. The combination of the two significantly reduces the false recognition and deception success rate. The lightweight pre-screening and hierarchical registration mechanism makes the system easy to deploy on embedded or edge devices, meeting the engineering needs of real-time monitoring and access control scenarios. The system's threshold management, online calibration, and environmental compensation functions enhance the stability and maintainability of cross-scenario and long-term operation. Attached Figure Description

[0067] The invention will now be described in more detail with reference to embodiments and the accompanying drawings.

[0068] Figure 1 shows a flowchart of the face recognition method based on sequential verification of point cloud data and RGB images provided in this application;

[0069] Figure 2 shows a flowchart of an RGB image verification method 200 according to an embodiment of the present disclosure;

[0070] Figure 3 shows the results of each sub-step in Method 200 for extracting face descriptors using the LBPH method and the CNN method, respectively.

[0071] Figure 4 shows the distribution of true and false sample scores based on cosine similarity S and the results of dual-threshold classification.

[0072] Figure 5 shows a flowchart of a point cloud geometry verification method 300 according to an embodiment of the present disclosure.

[0073] Figure 6 shows the spatial point cloud situation when performing voxel grid downsampling in each sub-step of step 301 of the method 300 of the present invention.

[0074] Figure 7 shows a schematic diagram of the experimental results of the local descriptor and robust coarse registration through step 301.

[0075] Figure 8 is a schematic diagram of the processing results of each sub-step of the point cloud geometry verification method of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] Figure 1 shows a flowchart of the face recognition method based on sequential verification of point cloud data and RGB images provided in this application. The overall process of this application is as follows: RGB images and point cloud data are acquired at the same time. Face detection, cropping, and feature extraction are performed on the RGB images and compared with the feature library to obtain the RGB verification similarity. Based on the relationship between the similarity and the first upper threshold and the first lower threshold, it is determined whether to directly enter the point cloud geometric verification or trigger the RGB image re-verification. When entering the point cloud geometric verification, geometric feature extraction and matching operations are performed on the point cloud region corresponding to the RGB face region to obtain the point cloud geometric verification similarity. Based on the relationship between the similarity and the second upper threshold and the second lower threshold, output information of verification pass, review, or rejection results is output, which corresponds to the verification result of whether the face recognition is the same person, not the same person, or uncertain. Among them, the identity result and confidence information are output only when the point cloud geometric verification meets the pass condition. When the RGB re-verification or point cloud re-verification still fails, the warning and fallback verification system is triggered to complete the identity confirmation. The threshold can be set statistically through the development set or the verification set, and can be adjusted in stages according to the application security level.

[0078] In the RGB image verification stage, face detection can be implemented using traditional detectors or deep learning detectors, such as Haar cascades, a combination of Histogram of Oriented Gradients (HOG) features and Support Vector Machine (SVM) classifiers, MTCNN, RetinaFace, YOLO, or SSD, all of which are used to output the face bounding box position. The cropping can be done by directly cropping according to the bounding box and scaling it to a fixed size (such as 128×128 or 224×224), or the boundary can be proportionally extended around the bounding box to cover areas such as the forehead and chin, and key point alignment (such as eye corner and nose tip alignment) can be combined to reduce pose differences. The image preprocessing can be any one or a combination of grayscale conversion and histogram equalization, CLAHE, gamma correction, white balance, illumination normalization, or Retinex enhancement to improve robustness under strong light, backlight, and shadow conditions. The RGB feature extraction can employ manual descriptors or deep feature embeddings, such as statistical features from any of the following: LBPH, HOG, Scale-Invariant Feature Transform (SIFT) features, or Oriented Fast and Rotated BRIEF (ORB) features; or embedding vectors obtained from networks such as ResNet, MobileFaceNet, ArcFace loss-trained face feature embedding networks, or IR-SE (Improved Residual with Squeeze-and-Excitation) face networks with improved residual structures and introduced compression excitation modules. To unify the calculation of RGB verification similarity, the feature vectors can be L2 normalized, and cosine similarity, Euclidean distance to similarity mapping, or similarity output based on classifier / metric learning models can be used as RGB verification similarity. Furthermore, the similarity can be converted into a confidence scale using the Sigmoid function or Platt calibration to ensure consistency with threshold management.

[0079] In the point cloud geometry verification stage, the point cloud data corresponding to the RGB face region can be used to enable the use of a depth camera through camera calibration and depth mapping. In some embodiments, the face bounding box region is mapped to a depth map or point cloud using the intrinsic and extrinsic parameters of the RGB and depth cameras, and the corresponding 3D point set is extracted. In other embodiments, 3D face / head segmentation can be performed in the point cloud first (e.g., based on depth thresholding, clustering, or a learned segmentation network), and then the intersection or consistency check is performed with the RGB detection results to determine the final face point cloud region. The point cloud preprocessing may include voxel downsampling, statistical outlier removal, radius filtering, normal estimation, and normal orientation unification to reduce noise and stabilize subsequent matching. The geometric feature extraction and matching operations can employ local descriptor matching or direct registration: for example, calculating local descriptors such as FPFH, SHOT, and Spin Image and performing nearest neighbor matching, then sampling and filtering interior points using either a robust interior point selection mechanism based on Random Sample Consensus (RANSAC) or a sampling selection mechanism based on consistency constraints (e.g., iterative selection based on correspondence consistency), combined with SVD closed-form solutions or nonlinear optimization (such as ICP, Point-to-Plane Iterative Closest). The optimal rigid body transformation is obtained by using the Point (point) registration algorithm, LM nonlinear least squares optimization, and Gauss-Newton (GN) optimization method. The point cloud geometric verification similarity can be constructed from the registration residuals, such as based on the quantile statistics of the point-to-surface normal orthogonal residuals, based on the proportion of interior points or the mean of the interior point residuals, or based on the set distance such as Chamfer distance. Robust scaling normalization can be performed on the residuals to suppress the influence of extreme values. Finally, the statistical quantity is monotonically mapped to obtain the point cloud geometric verification similarity in the interval [0,1], which is used to compare with the second upper threshold and the second lower threshold to output the pass, review, or rejection results.

[0080] Furthermore, the RGB image re-verification can be achieved through multi-frame resampling, changing sampling conditions (such as prompting the user to slightly turn their head, changing the distance, or supplementing lighting), or cross-verification using different feature extractors. Median, weighted average, and voting fusion can be applied to multiple similarities to reduce false acceptances without significantly increasing false rejections. The point cloud geometric re-verification can obtain a more complete face point cloud through continuous multi-frame point cloud fusion (such as voxel fusion or keyframe fusion), then repeat the geometric verification and perform ensemble-level fusion judgment on multiple results. If the RGB re-verification or point cloud re-verification still fails to meet the passing conditions, the recognition result is submitted to the warning and fallback verification system, triggering at least one of password verification, fingerprint verification, or manual review to ensure reliable identity verification can still be completed under conditions of poor lighting, severe occlusion, or increased sensor noise.

[0081] In some embodiments, the RGB image verification employs a tiered judgment mechanism to balance the efficiency determined by the verification and output of the verification result with the security level determined by the accuracy of the verification. Specifically, when the RGB verification similarity is higher than a first upper threshold, the RGB result is considered to have high credibility, and it directly proceeds to point cloud geometric verification for three-dimensional geometric consistency review; when the RGB verification similarity is lower than a first lower threshold, the RGB result is considered to have insufficient credibility, the RGB verification is judged to have failed, and the current recognition process is terminated or an alarm is triggered; when the RGB verification similarity is between the first upper threshold and the first lower threshold, the result is marked as ambiguous and enters the RGB re-verification process. The RGB re-verification process can be achieved by re-acquiring RGB images of the same target under different time frames, different lighting conditions, or different pose cues, and repeatedly calculating the RGB verification similarity. Statistical fusion methods such as median fusion, weighted average fusion, or majority voting fusion can be used to generate fusion results. When the fusion result is higher than the first upper threshold and each result is not lower than the first lower threshold, the point cloud geometric verification stage is allowed; otherwise, a rejection result is output, an alarm is triggered, or an auxiliary identity verification process is initiated to reduce the risk of misjudgment caused by single-frame noise, occlusion, or instantaneous lighting changes.

[0082] In some embodiments, the point cloud geometric verification also employs a hierarchical judgment mechanism to reduce noise inaccurate depth verification results and improve the robustness of local missing constraints. Specifically, when the point cloud geometric verification similarity is higher than the second upper threshold, the point cloud verification is deemed successful and a geometric verification conclusion of "same person" is output; when the point cloud geometric verification similarity is lower than the second lower threshold, the point cloud verification is deemed unsuccessful and a rejection or alarm message is output; when the point cloud geometric verification similarity is between the second upper threshold and the second lower threshold, the point cloud re-verification process begins. The point cloud re-verification process can be achieved by reacquiring point cloud data of the target corresponding region, or by extracting and fusing face point cloud sets from multiple consecutive point clouds and then performing point cloud geometric verification again. The similarity of multiple point cloud geometric verifications is judged by set-level fusion. When the set-level point cloud geometric verification similarity is higher than the second upper threshold within a specified number of times, it is considered passed. When the set-level point cloud geometric verification similarity is lower than the second lower threshold within a specified number of times, an alarm is triggered, an auxiliary identity verification process is entered, or a rejection result is output. Other cases are judged as uncertain and point cloud re-verification continues, or an alarm is triggered or an auxiliary identity verification process is entered after the number of times is exhausted. This reduces false rejections caused by missing point clouds, pose changes, or sampling jitter while maintaining a security threshold.

[0083] Figure 2 shows a flowchart of an RGB image verification method 200 according to an embodiment of the present disclosure. In step 201, the input samples are multiple color face images. The values ​​for the color face image range from 0 to 255 (8 bits), where H, W, and 3 represent the number of rows (pixel height), columns (pixel width), and channels of the color face image, respectively.

[0084] In some embodiments, step 201 specifically includes the following steps:

[0085] Step 2011: For each input RGB face image, a face detector is used to detect the face border, and the face region is cropped based on the face border and then scaled to a fixed size image with a preset side length.

[0086] Step 2012: Perform image equalization processing on the cropped and scaled face image to obtain an equalized image. When using LBPH to extract face descriptors, the cropped and scaled image is first converted to grayscale before histogram equalization or contrast-limited adaptive histogram equalization (CLAHE algorithm) is performed.

[0087] Step 2013: Perform illumination compensation on the equalized image to obtain an illumination-compensated image. The illumination compensation includes nonlinear correction of pixel intensity based on gamma coefficients. When the contrast adaptive histogram equalization (CLAHE algorithm) is limited and a preset enhancement intensity is reached, the illumination compensation step can be omitted to avoid over-enhancement.

[0088] Specifically, in step 2011, for each input color image, a face detector is used to detect the border. The image is further cropped and scaled to a square image with a fixed side length of M. The area of ​​the cropped and scaled square image is... (For example The cropped and scaled image in pixel coordinates Pixel value of channel c for:

[0089] ;

[0090] in, , , ; , R, G, and B are the x and y coordinates of the cropped and scaled output image, respectively, both of which are integers. R, G, and B are the red, green, and blue channels of channel c, respectively.

[0091] In existing technologies, face detectors can employ traditional Haar cascades (Viola–Jones method), classifiers combining HOG and SVM (e.g., implemented using dlib), or deep learning detectors (e.g., MTCNN, RetinaFace, YOLO, or SSD); all of these methods can output face bounding boxes. The image preprocessing method disclosed herein does not depend on the internal implementation of a specific detector, and is used for subsequent cropping and feature extraction. In the bounding box U, x and y are the horizontal and vertical coordinates (in pixels) of the upper left corner of the face bounding box, respectively, and w and h are the width and height (in pixels) of the face bounding box, respectively.

[0092] In step 2012, the cropped image undergoes image equalization processing. In some embodiments, if LBPH is subsequently used for face description, the cropped image needs to be equalized. Image equalization is performed only after grayscale conversion. The grayscale conversion yields... :

[0093] ;

[0094] This step uses a fixed weighting to calculate the pixel values ​​after cropping and scaling. The output image is synthesized into a single-channel brightness, which is ultimately represented by the grayscale value of the pixel in the output image. .

[0095] Histogram equalization is performed on the cropped and scaled image after grayscale conversion to obtain the grayscale histogram. , This represents the number of pixels that satisfy the condition that the gray level is equal to k, where k = 0, 1, 2, ..., 255; for the histogram The cumulative distribution of each gray level i in the histogram is statistically analyzed to obtain the cumulative distribution of the histogram. i=0,1,…,k; and then, after histogram equalization, a balanced grayscale image is output. :

[0096] .

[0097] In step 2013, the equalized grayscale image after equalization is further subjected to illumination compensation to obtain a compensated pixel image. :

[0098] ;

[0099] in, γ is the gamma correction coefficient (dimensionless), used to control the grayscale image processed during illumination compensation. The nonlinear compensation intensity is such that the larger γ is, the smaller 1 / γ is, the dark part is moderately lifted and the bright part is moderately compressed to reduce the influence of illumination.

[0100] In other embodiments, step 2011 directly performs CLAHE equalization (limited contrast adaptive histogram equalization), dividing the image into... Small blocks are processed, and contrast histogram equalization is applied to each block. The blocks are then stitched together using bilinear interpolation to generate an enhanced result (avoiding global overstretching).

[0101] In this embodiment, the CLAHE algorithm has already performed local contrast enhancement, and the illumination compensation in step 2013 is a mild global brightness remapping. If the image enhancement intensity after processing by the CLAHE algorithm is sufficient, the illumination compensation process in step 2013 can be omitted to avoid over-enhancement.

[0102] In some embodiments, to improve the stability of subsequent descriptor extraction, a global Otsu segmentation can be optionally performed once after grayscale, equalization, and illumination compensation (after step 201) and before proceeding to step 202 to obtain the threshold. And generate a foreground mask , For indicator functions, when The function returns 1 if the condition within it is true, and 0 otherwise. Subsequently, only when the condition within it is true... Feature statistics are performed on pixels: For LBPH, only the foreground region's grid histogram is counted to suppress background texture interference; for CNN, hard segmentation is not necessary, or... This is used as a weight (or the original image is preserved) to avoid losing details. This reduces background noise without changing the calculation and thresholding framework of subsequent face extraction sub-processing steps.

[0103] In some embodiments, step 202 employs the LBPH method and includes the following steps:

[0104] Step 2021: Calculate the Local Binary Pattern (LBP) code pixel by pixel for the preprocessed face image using a preset radius and a preset number of neighborhood samples;

[0105] Step 2022: Divide the face image into a preset number of sub-grids, and calculate and normalize the LBP encoded histogram for each sub-grid;

[0106] Step 2023: Concatenate the normalized histograms of each subgrid in order to obtain the LBPH feature vector, and perform L2 normalization on the LBPH feature vector to obtain the unit feature vector;

[0107] Specifically, in step 2021, when using the LBPH method, the LBP code of each pixel is first calculated with a radius r of 1 and a neighborhood of 8:

[0108] ;

[0109] in , where g is the independent variable of the s() function, and s(·) is the step function used to compare the size relationship between neighboring pixels and the center pixel. , respectively with Center, radius , No. One neighbor ( In polar coordinate direction The corresponding sub-pixel x and y coordinates; For coordinates Image after illumination compensation correction The pixel intensity values ​​obtained by bilinear interpolation. The result of bilinear interpolation is the image. In non-integer coordinates The pixel grayscale value at a location, in non-integer coordinates The pixel grayscale value at that location is determined by It is obtained by weighting the surrounding four integer pixels using bilinear weights.

[0110] , .

[0111] In step 2022, Divide evenly A grid, in some embodiments an 8×8 grid, is used to compute the 256-dimensional histogram of the g-th subgrid in that grid. Normalize the 256-dimensional histogram of the g-th sub-grid to obtain the 256-dimensional normalized histogram of the g-th sub-grid. :

[0112] ;

[0113] in, for The L1 norm, i.e. After normalization, the sum of the coefficients of each grid histogram is 1. The meaning is that it satisfies the LBP code of the pixels within this grid. equal The number of pixels.

[0114] In step 2023, the total The normalized histograms of each subgrid are sequentially concatenated to obtain a length of [missing value]. column vectors :

[0115] ; ;

[0116] The normalized feature scale is then obtained through further normalization. : ;

[0117] in, yes The L2 norm, that is By using L2 normalization, the vector length is made to be 1, which is convenient for cosine similarity comparison.

[0118] In some other embodiments, when step 202 employs a CNN method, it includes the following steps:

[0119] Step 2021: Perform pixel normalization on the preprocessed face image. Normalize the pixels based on the channel mean and channel standard deviation statistically obtained from the training set to obtain the network input tensor.

[0120] Step 2022: Input the network input tensor into the trained convolutional neural network and perform forward inference to obtain the embedded feature vector, and perform L2 normalization on the embedded feature vector to obtain the unit feature vector;

[0121] The unit feature vector serves as the input for subsequent similarity calculations, enabling different descriptor extraction methods to be compared uniformly under the same threshold judgment framework.

[0122] Specifically, in some other embodiments, in step 2021, before CNN processing, the pixel image after illumination compensation in step 2013 needs to be normalized to obtain a normalized pixel image. :

[0123] ;

[0124] in, , These are the pixel mean and pixel standard deviation of channel c on the training set after undergoing the same preprocessing.

[0125] In step 2022, when choosing to use a CNN method to extract face descriptors, a trained network (e.g., ResNet-50 or IR-SE) is used to... The forward pass yields the penultimate layer embedding graph. :

[0126] ;

[0127] d = 128 or 512; d is the dimension of the feature vector. For the feature extraction mapping (forward inference function) of the trained CNN model, the input is the normalized image tensor, and the output is the feature vector of the penultimate layer or the embedding layer. .

[0128] Further analysis of the penultimate layer embedding graph Normalization process yields normalized feature scales. :

[0129] ;

[0130] Both the LBPH method and the CNN method extract face descriptors, ultimately outputting isomorphic unit vectors: normalized feature scales. When extracting facial features using the LBPH method, When extracting facial features using CNN methods, This facilitates standardized similarity calculation and threshold determination.

[0131] In some embodiments, step 203 involves calculating the normalized feature scale to be measured. Compared with reference feature scale The similarity between the two can be calculated by using the cosine similarity result as the RGB verification similarity or by using the equivalent mapping from Euclidean distance to similarity. The specific steps include:

[0132] Step 2031, calculate cosine similarity :

[0133] ; ;

[0134] The equivalent mapping from Euclidean distance to similarity is as follows:

[0135] calculate and Euclidean distance between them: ,

[0136] , Therefore, the similarity of Euclidean mappings Cosine similarity to the above calculation method Different paths lead to the same destination.

[0137] Step 2032, construct the confidence calibration model Conf for RGB verification similarity calibration:

[0138] ;

[0139] , The first and second factors, respectively, are used to calibrate the confidence level. These factors can be obtained by combining the minimum cross-entropy values ​​on the development set. After training and statistics, in some embodiments, the optimal factor is taken. , . It is the logical sigmoid function.

[0140] RGB similarity verification ( The work order is calibrated using the Conf confidence scaling model to obtain a normalized confidence scale, which facilitates integration, display, and unified threshold management with scores from other modules. The Conf result is approximate; the closer the Conf value is to 1, the more reliable the similarity is for subsequent comparisons with the first upper threshold and the first lower threshold. The calibrated similarity is then compared with the subsequent first upper threshold and the first lower threshold to determine the execution of the next process step.

[0141] As shown in Figure 3, the sub-images in the first row are step-by-step processing images processed by the above-mentioned method of this application through grayscale conversion, equalization, illumination compensation and adaptive thresholding.

[0142] The first row shows the sub-images. The first sub-image is the original color image with borders detected and marked by the face detector. The second sub-image is the grayscale image processed in step 2012, which removes interference from color components, expressing facial structure only through brightness, and making the contours of eyebrows, eyes, and nose clearer. The third sub-image is the result of further histogram equalization of the grayscale image. Compared to the second sub-image, the third sub-image shows that the dark areas of the face are enhanced, the bright areas are compressed, the details are revealed, and the contrast of facial textures is significantly improved. Then, illumination compensation in step 2013 (with a gamma correction factor of 2.2) is performed to obtain the fourth sub-image. Compared to the third sub-image, the dark areas of the face in the fourth sub-image are further enhanced, the facial information in the shadow areas is more complete, the bright areas are not overexposed, and the overall illumination tends to be more balanced. Based on this, Otsu automatic thresholding is used to generate a binary image (the fifth sub-image). It can be seen that the foreground and background are stably separated, and the facial contours and facial feature areas are blocky and coherent, which is convenient for subsequent texture counting. After further processing in steps 2021-2023, the resulting LBP texture map exhibits stable bands and patches of high-frequency details such as eyebrow and brow wrinkles, and nasolabial folds, with continuous edges and insensitivity to changes in illumination. This sequence tightly integrates brightness remapping with adaptive thresholding, ensuring that the LBP grid histogram remains comparable under different lighting and background conditions, thus providing stable cosine similarity and confidence calibration results under a unified threshold.

[0143] The sub-images in the second row of Figure 3 represent the stable input distribution processed by the method described in this application. This process involves grayscale conversion, equalization, illumination compensation, pixel normalization, and deep embedding. The stable input distribution is then processed using a CNN method to extract face descriptors, ensuring the CNN feature vectors maintain a consistent scale under different shooting conditions. The first sub-image in the second row is the original color image already detected and labeled with borders using a face detector. The second sub-image represents the grayscale image processed in step 2012. Compared to the first sub-image, the second sub-image first eliminates color cast factors, retaining only geometric and texture information. After histogram equalization, the subtle facial structures are clearly visible, and the contrast between hair and skin texture is expanded. Continuing with the illumination compensation in step 2013, the third sub-image is obtained. Compared to the second sub-image, the facial details in the shadow areas of the third sub-image are gently brought out, while the highlights are suppressed, and the overall exposure is closer to the statistical distribution of the network training set. Unlike the first row, which uses the LBPH method to extract facial descriptors, this CNN extraction method does not perform binarization. Instead, it preserves grayscale details and performs channel normalization to obtain the fifth sub-image. Channel normalization aligns the pixel distribution to the mean and variance during training, and then inputs it into the trained CNN convolutional network to extract embedded features, resulting in the final CNN facial descriptor extraction result (the sixth sub-image). This method effectively reduces representation drift caused by device differences and scene exposure differences, making the unit vector of the penultimate layer more stable across scenes and devices, facilitating direct integration with a unified thresholding framework.

[0144] Figure 4 shows the distribution of true and false sample scores and the results of dual-threshold classification based on cosine similarity S. As shown in Figure 4, simulation tests were conducted on steps 201-203 of this application using both fake and genuine samples, and the results were obtained based on cosine similarity S. The scoring distribution uses a double threshold system with a first upper threshold and a first lower threshold (the first upper threshold...). =0.78, first lower threshold When making a classification decision based on (=0.48), the total sample is divided into those that directly pass the decision interval (greater than 0.48). Region), re-verification trigger interval (less than or equal to) and greater than or equal to (region) and direct rejection judgment interval (less than) area).

[0145] Of the genuine samples, 78.33% directly entered the pass / fail decision interval, 21.67% fell into the revalidation trigger interval, and 0.00% were directly rejected; of the imposter samples, 65.00% directly entered the direct rejection decision interval, 35.00% fell into the revalidation trigger interval, and 0.00% mistakenly entered the pass / fail decision interval. From a single threshold perspective, in The system yields a security-availability combination of FAR = 0.000% and FRR = 21.667%. FAR stands for False Acceptance Rate, and FRR stands for False Error Rate. Given a decision threshold, FAR is the percentage of similarity scores (greater than or equal to the threshold) of all imposter samples out of the total number of imposter samples. FRR is the percentage of scores (less than the threshold) of all genuine samples out of the total number of samples. Under the prior assumption of a 1:1 ratio (50% genuine samples and 50% imposter samples), the expected percentage of samples entering the revalidation trigger interval is approximately 28.33%. The results show that by setting upper and lower thresholds to divide the results of the first stage (i.e., the RGB face verification stage) into a direct pass decision interval, a direct rejection decision interval, and a re-verification trigger interval, the potential false rejection samples can be mainly placed in the re-verification stage while maintaining a near-zero false acceptance rate in the RGB face verification stage. The workload of the point cloud geometric verification stage is controlled to a manageable 28%. This proves that the sequential verification method 200 of this disclosure, which performs the RGB face verification stage and then the point cloud geometric verification stage, first delineates a clear direct pass decision interval and a re-verification trigger interval in the similarity space with a first upper threshold and a first lower threshold. Samples in the re-verification trigger interval are further transferred to the point cloud geometric verification stage, and the complementary information of different modalities can be used to improve the discriminativeness.

[0146] In some embodiments, RGB re-verification can re-acquire multiple face images of the same target at adjacent time points or under improved sampling conditions (supplementary lighting results, alignment results, or occlusion removal results), compare them one by one according to the original preprocessing and feature extraction process, and then perform multiple result fusions (the result with consistent mode, median, and average can be taken as the fusion result). If the first upper threshold is reached multiple times in a row, it is determined to pass; otherwise, it is switched to alarm or auxiliary verification.

[0147] Figure 5 shows a flowchart of a point cloud geometry verification method 300 according to an embodiment of the present disclosure. Method 300 includes the following steps:

[0148] The input source point cloud is Reference point cloud used for training and subsequent evaluation , Let α be the three-dimensional coordinate vector of the α-th source point. Let be the three-dimensional coordinate vector of the β-th reference point; source point cloud Reference point cloud All are physical and mental sets. α = 1, 2, ... ;β=1,2,…, .

[0149] In step 301, voxel downsampling, outlier removal and normal estimation are performed on the source point cloud and the reference point cloud to ensure that the two sets of point clouds meet the stability requirements of subsequent matching and consistency evaluation in terms of point density, noise and local geometric representation.

[0150] In step 302, the local geometric descriptor of the preprocessed point cloud is obtained and a point-to-point correspondence is established. An iterative robust estimation method is used to select the set of interior points that meet the error constraints from the candidate correspondence. The optimal rigid body transformation parameters are obtained by minimizing the registration residual based on the set of interior points, so as to achieve robust registration from the source point cloud to the reference point cloud.

[0151] In step 303, the normal orthogonal residual is calculated based on the optimal rigid body transformation parameters, and robust normalization and statistical aggregation are performed on the residual to obtain the point cloud geometric verification similarity. The point cloud geometric verification similarity is compared with the second upper threshold and the second lower threshold, and the verification pass, review, or rejection result is output. Passing verification means that the verification result is that both RGB verification and point cloud verification are passed, and the verification result output is "same person" for face recognition. If the point cloud verification fails, but the RGB verification passes, the corresponding "review" output information is provided above, and the system determines that it is "uncertain" whether it is the same person, and point cloud re-verification is required. If the point cloud verification fails, and both RGB verification and / or RGB re-verification fail, the corresponding "reject" output information is provided above, and the system determines that it is "not the same person".

[0152] In some embodiments, step 301 specifically includes the following steps:

[0153] Step 3011: Use the preset voxel side length to perform voxel grid downsampling on the source point cloud and the reference point cloud, and replace the points falling in the same voxel with their centroid points to obtain the voxel centroid set after downsampling.

[0154] Step 3012: For the point cloud after voxel grid downsampling, calculate the average Euclidean distance from each point to its neighboring points within a neighborhood of a preset number of nearest neighbors, and identify and remove points whose average Euclidean distance exceeds the threshold determined by the global mean and standard deviation as outliers.

[0155] Step 3013: For the point cloud after removing outliers, construct a local three-dimensional spherical neighborhood with a preset neighborhood radius or a preset number of neighborhood points, form a covariance matrix based on the neighborhood point set and perform eigenvalue decomposition, and take the eigenvector corresponding to the smallest eigenvalue as the estimated normal vector of the point.

[0156] In step 301, the input source point cloud is Reference point cloud used for training and subsequent evaluation , Let α be the three-dimensional coordinate vector of the α-th source point. Let be the three-dimensional coordinate vector of the β-th reference point; source point cloud Reference point cloud All are physical and mental sets. α = 1, 2, ... ;β=1,2,…, .

[0157] Specifically, in step 3011, firstly, each direction in the three-dimensional space is defined with respect to the voxel side length. The parameters are used to perform voxel grid downsampling on each point in the source and reference point clouds:

[0158] ;

[0159] in, A 3D point vector in a point cloud (which can come from the source point cloud). ),Right now , Let be the average coordinate vector of the centroids of all points within the voxel (a, b, c), where a, b, and c are the points... The voxel indices along the X, Y, and Z axes in 3D space are all integers, representing the starting coordinates of the voxel raster downsampling. Then the side length is cubic voxels for:

[0160] ;

[0161] For a three-dimensional coordinate vector any point Its three-dimensional index is:

[0162] ;

[0163] in, This is the floor function.

[0164] To replace the original point cloud with the set of centroids of each occupied voxel (i.e., to complete downsampling), the reference point cloud also needs to be processed during training and use of this method. Perform this operation.

[0165] In step 3012, the points within the point cloud are counted. The number of nearby is The neighborhood formed by the nearest points Anomalies within the data are identified and removed. A formula for calculating the removal criteria is then constructed.

[0166] ;

[0167] like Then determine the points within the point cloud. For outliers, the point is... Remove.

[0168] Remove the judgment formula To count the number of neighboring points when there are outliers in a local neighborhood; Set a threshold for removing outliers. For point To neighboring areas The average Euclidean distance between the nearest neighbors , These are the global mean and standard deviation of the average Euclidean distance between all points within the source point cloud, respectively. It is the coordinate vector of a single neighboring point within the neighborhood. Represents the neighboring domain A single neighboring point within .

[0169] In step 3013, using points The neighborhood radius is centered at the normal and estimated as . , or by point The number of neighborhood points centered at the circle and whose normal is estimated to be... The formed three-dimensional spherical domain Normal estimation is performed on the set of points within the range, and the normal estimation is calculated for the neighborhood of the points within the range. Adjacent points covariance matrix :

[0170] ;

[0171] For covariance matrix By estimating the normal vector, we obtain the estimated normal vector of point x. :

[0172] ;

[0173] in, for The transpose of the matrix, For points The three-dimensional mean coordinates of adjacent points, i.e., the coordinates of the points. The three-dimensional coordinates of a point are determined by averaging the three-dimensional coordinates of all adjacent points. The centroid of the neighborhood point, i.e., point For point Limit the normal estimation neighborhood (i.e., the three-dimensional sphere) The center point of ) For statistical analysis of three-dimensional spherical domains Interior and Point Points adjacent to each other The number function, ; This is the operator that takes the eigenvector corresponding to the smallest eigenvalue after performing eigenvalue decomposition on a matrix, i.e., for the covariance matrix. Perform eigenvalue decomposition and take the smallest eigenvector as the normal.

[0174] covariance matrix This describes the second-order spatial distribution of neighborhood points in three directions. The covariance matrix... By performing eigenvalue decomposition, the eigenvector corresponding to the smallest eigenvalue gives the normal direction of the best-fit plane for that local point cluster. In this way, even in the presence of noise, disturbances can be suppressed by statistical averaging, resulting in a stable and reliable local surface normal. This provides directional information for subsequent error statistics from point to surface and for normal consistency constraints.

[0175] In step 301, the normal refers to the point The unit outward normal vector of the local surface, not the direction of the voxel center. Its function is to provide stable geometric direction information for subsequent point-to-surface errors, normal consistency constraints, and descriptors (such as FPFH), thereby accelerating registration convergence and reducing mismatches. In principle, in a point-to-surface context... Estimating the neighborhood radius using the normal to the center Calculate the covariance matrix within the formed three-dimensional spherical domain The eigenvector corresponding to its smallest eigenvalue This refers to the normal to a local plane (or a differentiable surface); the orientation of the normal is then unified through a simple unification strategy (e.g., the dot product of the line connecting the viewpoint or centroid is positive). Because for points... Estimating the normals can statistically represent and quantify the point cloud position of each point in the source point cloud by combining position and orientation information. Registration can be improved from point-to-point to point-to-surface and introduce a normal consistency term, which can usually significantly improve robustness to noise and sparse sampling.

[0176] As shown in Figure 6, this step first downsamples the original point cloud shown in Figure 6(a) by voxel grid: Figure 6(a) shows that, in the simulated point cloud of this embodiment, the size of the original point set is the number of unvoxed points N. raw =6300, step 3011 divides the three-dimensional space into sections with side lengths of... =12mm cube voxel and in formula The center of the voxel is used to replace all points within the voxel, thus obtaining the spatial domain after voxel grid downsampling as shown in Figure 6(b). After voxel grid downsampling ( After obtaining the number of points N after voxel grid downsampling (=12mm), the result is the number of points N after voxel grid downsampling. voxel=2589, a 58.90% reduction in the number of points (each occupied voxel is represented by a centroid point), significantly reducing the computational scale of subsequent nearest neighbor and matching operations while maintaining the overall shape. This demonstrates that by using voxel grid downsampling in step 3011, the number of points can be compressed to the order of occupied voxels, enabling geometric control of each point to the original surface within the acceptable error range. Within the specified range, sampling is performed uniformly while preserving the overall geometry. Then, step 3012 is performed to identify and remove outliers, calculating the value for each point. Average distance of nearest neighbors Then, a statistical threshold is defined by the global mean and global variance. Outlier removal is performed, as shown in Figure 6(c), by applying statistical outlier removal to the voxelization results. =24, The result obtained after (=1.6) is the number of points N remaining after further counting and removing outliers based on the voxelization result. sor =2352, the number of points in the set after voxelization is reduced by 9.15%, and the overall retention rate is 37.33%, effectively removing outliers. Unlike methods with fixed radius or fixed threshold, step 3013 constructs a removal criterion, allowing the statistical threshold to adapt to local density, thus robustly suppressing fly-point and scattered point noise. Finally, normal estimation is performed on the denoised point set, and a covariance matrix is ​​constructed for each query neighborhood. Take the eigenvector corresponding to the smallest eigenvalue. The normal direction at that point is used as the normal direction, and the dot product with the scene center is used for consistent orientation, thereby obtaining a stable surface orientation field under noise constraints.

[0177] Further, as shown in Figures 6(a)-6(d), from the perspective of geometric density indices, the k-nearest neighbor average distance distribution index of the voxel cloud has a mean of 0.0314m and a standard deviation of 0.0326m. After outlier statistics and removal in step 2012, the k-nearest neighbor average distance index of the voxel cloud shrinks to a mean of 0.0216m and a standard deviation of 0.0060m, indicating that the local density is more consistent and the variance is significantly reduced, creating better neighborhood conditions for robust registration. Step 3013 in the three-dimensional sphere... Normal estimation (k) within the interior n When =30), As a proxy indicator of directional consistency ( (where is the normalized radial vector of the point), the k-nearest neighbor average distance index of the voxel cloud after processing in step 3013 is: mean 0.9493, standard deviation 0.0413, indicating that the normal direction is consistent with the height of the outward normal of the surface.

[0178] Therefore, based on step 301 above, the input source point cloud is preprocessed. Step 3011, voxelization, reduces the scale of subsequent nearest neighbor search and dimension matching from the original total number to the number of occupied voxels (usually the number of occupied voxels is significantly smaller than the original total), thus simultaneously reducing the time and memory complexity of subsequent precise matching. Step 3012 statistically analyzes the neighborhood. Anomalies within and , and The determined statistical threshold is used as a criterion to remove outliers, thereby effectively controlling the pass rate of outliers in this step and significantly reducing the risk of incorrect correspondences and local extrema in registration; in step 3013, the three-dimensional sphere is defined. The normals are estimated within the sphere, providing stable normal constraints for subsequent error control and outlier removal suppression that does not meet the requirements for subsequent point cloud geometric verification. Furthermore, it provides reliable priors for the subsequent fine registration target of "point to surface + normal consistency".

[0179] It should be noted that the range of values ​​for the three-dimensional coordinate axes in Figure 6 is [−0.3, 0.3] (the unit is m, representing the world coordinate range of the point cloud, while the voxel size is...). =12mm refers to the granularity of binning in this three-dimensional space using a cubic mesh with a side length of 12mm; the two are concepts of different dimensions, and how the coordinate range is set does not affect the correctness and reproducibility of voxel binning and subsequent statistical denoising, normal estimation and registration results.

[0180] In some embodiments, step 302 includes the following steps:

[0181] Step 3021: After completing the preprocessing, local descriptors of the source point cloud and the reference point cloud are calculated in the preset statistical neighborhood, and the local descriptors are normalized to obtain local feature vectors.

[0182] Step 3022: Based on the local feature vector, determine the corresponding point with the smallest descriptor distance for each source point in the source point cloud in the reference point cloud, and establish a candidate matching pair set between the source point and the corresponding point;

[0183] Step 3023: Perform robust consistency screening on the candidate matching pair set to retain inlier matching pairs that satisfy the local description pair retention threshold constraint, construct a residual minimization model based on the inlier matching pair set and solve for the optimal rigid body transformation parameters from the source point cloud to the reference point cloud.

[0184] After completing step 301 of preprocessing, the processed source point cloud... Compared with the processed reference point cloud Calculate the local geometric description and obtain the initial pose. =1,2,…, ; =1,2,…, . For the first step after preprocessing in step 301 The three-dimensional coordinate vector of each source point For the first step after preprocessing in step 301 A three-dimensional coordinate vector of a reference point.

[0185] In step 3021, using the normal as a priori, the neighborhood radius is statistically analyzed according to the local description. Under constraints of quantization binning θ=11 and feature dimension Q=3θ=33, local descriptors within the statistical neighborhood are statistically analyzed. Local descriptors are calculated according to the FPFH standard and then L2 normalized to obtain the FPFH local descriptors of the source point cloud. During training, the FPFH local descriptor of the reference point cloud is obtained. , , ;

[0186] In step 3022, within the local descriptive statistical neighborhood, the nearest neighbor is found for each point in the source point cloud. To obtain a local description ; ;

[0187] In step 3023, 2×10 iterations are performed on the outer layer of RANSAC. 3 ~2×10 4 Each iteration generates candidate rigid body coordinates using a small amount of local description. And use interior points to determine whether the following locally reserved description pairs of inequalities are satisfied:

[0188] ;

[0189] For local description of the retention threshold, Units are in mm; rigid body coordinates In It is a translation vector. It is a three-dimensional rotation matrix.

[0190] Local description pairs that satisfy the above inequalities are retained as interior point matching pairs, and multiple interior point matching pairs form the optimal interior point set. That is, the set of interior point matching pairs The optimal rigid body transformation parameters are solved by using a weighted SVD minimization model on points within the optimal interior point set. :

[0191] ;

[0192] in, These are weighting coefficients. , For local description of points Nearest Neighbor The Euclidean distance; For residuals, ; The unit is mm. The function is the Luban kernel, specifically the Huber or Tukey Luban kernel. The weighted SVD minimization model employs weighted singular value decomposition least squares. Given the correspondence between point pairs, singular value decomposition is performed on the weighted cross-covariance matrix to obtain the optimal rotation. Then, translation is calculated using the weighted centroid, thereby minimizing the weighted squared error of each corresponding point in a closed loop.

[0193] This step establishes robustness to pose and sampling density by adhering to the FPFH standard, then uses RANSAC iterative optimization to reduce the proportion of erroneous local description pairs. Finally, in the weighted SVD minimization model, the weighting coefficients obtained by solving the robust kernel are used. In the optimal rigid body solution process, the influence of residual outliers is further suppressed. The final closed-form solution minimized by weighted SVD ensures computational efficiency and result stability, thereby providing high-quality initial values ​​with a larger convergence domain for subsequent fine registration and significantly reducing the risk of divergence caused by mismatches.

[0194] In rigid body transformation parameters middle, , The group consisting of all rotations from the origin of three-dimensional Euclidean space is also called the rotation group; matrix satisfy , ( (to calculate the rank of the determinant of a matrix). The source point, after being processed in step 301, is positioned using a rigid body. Mapped to the reference coordinate system Minimize using weighted SVD The obtained optimal rigid body transformation parameters Rigid body registration is the process of minimizing the weighted squared residuals between the transformed source point cloud and the reference point cloud: all source points are registered according to... Align to reference point for robust weights Suppressing outlier correspondences. The purpose of solving for the optimal rigid body is to strictly bring the two clouds into the same coordinate frame and obtain stable and measurable geometric consistency (such as residuals, inlier rate, point-to-surface error, etc.), thereby providing a unique, reproducible, and noise-resistant alignment benchmark for subsequent point cloud similarity calculation and second upper and lower threshold determination.

[0195] Figure 7 shows a schematic diagram of the experimental results of local descriptor and robust coarse registration through step 301. In step 301, the preprocessed source point cloud is first... With reference point cloud Calculate their FPFH local geometric descriptors respectively, while satisfying the statistical neighborhood radius. mm, number of quantitative bins Feature Dimension Under parameter constraints, the local descriptor of each point can simultaneously reflect the normal distribution characteristics and neighborhood geometric structure features of the point. This is achieved by applying parameters to the source point in the descriptor space. FPFH vectors and all reference point clouds Perform nearest neighbor search on the FPFH vectors to establish a unique local description pair for each source point. This process forms candidate local geometric matching relationships. In Figure 7, the lines connecting the source point and the reference point represent these local description pairs. Solid lines represent highly consistent matching pairs determined as interior points by RANSAC iteration, while dashed lines represent potentially mismatched pairs that failed the consistency test. Interior points and reference points connected by solid lines are circled in black. After obtaining the initial set of local description pairs, a small number of sample pairs are repeatedly selected from the local description pairs using the outer random sampling mechanism of RANSAC to construct candidate rigid body transformation parameters. And based on the consistency judgment condition

[0196] Validate all local description pairs, where A threshold is retained for local description. Inequality matching pairs that satisfy this constraint are determined to be interior points and included in the interior point set. In Figure 7, the interior points circled in black are those determined to satisfy the above constraints. The light blue shaded area in Figure 7 represents the spatially concentrated distribution region formed by these interior source points in the source point cloud. It intuitively reflects the highly reliable matching subset that maintains geometric consistency after RANSAC constraints, and serves as an effective data basis for subsequent optimal rigid body transformation parameter estimation in the interior point set. Based on this, a weighted singular value decomposition least squares method is introduced, and weight coefficients are constructed through a robust kernel function. For matching pairs with large residuals, weight suppression is applied to make the final solution process insensitive to residual outliers. The objective function for weighted SVD minimization is constructed as follows: The optimal rigid body transformation parameters are obtained using the weighted SVD closed-form solution. The geometric relationship formed by the "×" and "+" symbols and the arrows between them in the figure represents the spatial mapping direction between the centroids of the source point cloud and the reference point cloud. It is used to intuitively depict the overall geometric trend of the rigid body transformation, that is, the process by which the source point cloud aligns with the reference point cloud as a whole after rotation and translation.

[0197] Through the above steps 3021 (obtaining the FPFH local descriptor), 3022 (matching to obtain local neighbor pairs formed by the nearest points), and 3023 (using RANSAC outer layer iteration to obtain multi-layer robust constraints for interior points that meet the constraints), step 302 can still stably select highly consistent geometric matching pairs under pose changes, uneven sampling density, and local occlusion conditions. This effectively suppresses the interference of erroneous local descriptor pairs on pose estimation and achieves a good balance between computational efficiency and stability. This provides an initial pose with a large convergence region and high reliability for the subsequent fine registration stage, giving the entire point cloud registration process good repeatability, noise resistance, and numerical stability. The horizontal and vertical axes in Figure 7 represent the horizontal and vertical coordinates of the point cloud on a two-dimensional schematic plane, used to visually display the source point. With reference point The relative positional relationships, local paired connections, and rigid body transformation trends are shown. The coordinate range of approximately [-2, 3] and [-1.5, 2.0] is used to automatically reserve the display window after demarcating the boundaries based on the distribution of all drawn points in this schematic diagram, with the aim of fully presenting the geometric structure and connection relationships; therefore, the diagram... , The coordinate values ​​shown are merely illustrative representations of relative positions and do not correspond to the absolute physical coordinates or actual dimensional values ​​of the real point cloud. It should be noted that the relative spatial distribution of points in the figure is not arbitrarily constructed, but rather derived from the actual geometric relationship between the source and reference point clouds in the registration results of real experimental point clouds. This relationship has been schematically processed through normalization, translation, scale compression, and two-dimensional projection before being mapped to this coordinate system for display. This process preserves the authenticity of the neighborhood structure, pairing relationships, and overall rigid body alignment trend in the original data, while also improving the readability and clarity of the graph.

[0198] In some embodiments, step 303 includes the following steps:

[0199] Step 3031: Based on the obtained optimal rigid body transformation parameters, perform rigid body transformation on the source points in the source point cloud and determine their corresponding points in the reference point cloud. Calculate the normal orthogonal residual of the transformed source point relative to the local tangent plane of the corresponding reference point using the estimated normal vector of the corresponding point.

[0200] Step 3032 performs robust scaling estimation and scaling normalization on the normalized orthogonal residuals, and extracts upper quantile statistics from the normalized residual set to suppress the influence of heavy-tailed distributions and extreme outliers on the statistical results.

[0201] Step 3033: Construct the point cloud geometric verification similarity index by the upper quantile statistics through monotonically nonlinear mapping, and compare the point cloud geometric verification similarity index with the second upper threshold and the second lower threshold to determine whether the point cloud geometric verification is passed, enters re-verification, or directly outputs a rejection result.

[0202] When evaluating the geometric consistency of point clouds, the simple point-to-point distance is easily affected by sampling density and tangential misalignment, which can amplify the error. Therefore, this step prioritizes the use of point-to-surface (normal) residuals, so that the error is measured only by the normal orthogonal component relative to the reference surface, thereby suppressing the interference of tangential offset and more objectively characterizing the degree of geometric fit.

[0203] In step 3031, the optimal rigid body transformation parameters obtained in step 302 are... Find the index of its corresponding reference point cloud. Then, according to step 3013, the estimated normal vector of the point corresponding to the index in the reference point cloud is determined. Calculate the first... Point-to-surface normal orthogonal error between the source point and the reference point :

[0204] ;

[0205] in, ; for The transpose of the matrix;

[0206] Step 3031 transforms the source point into a point after rigid body transformation. + To its corresponding point on the reference point cloud The orthogonal distance of the local tangent plane. The estimated normal vector of the corresponding point is used in the calculation. The inner product is calculated. The unit is meters. It only measures the component of the error in the normal direction (positive and negative indicate the direction on either side of the normal), and does not take into account tangential slip. Therefore, compared with the distance from the source point to the reference point as an evaluation index, it reflects the true fit between the two clouds on the local surface more accurately, converges faster, and has less impact on sampling density, thus making the final judgment result of tangential mismatch less affected. In practical use, its absolute value is often taken. or square To perform statistics, the orientation of the normals must first be standardized (e.g., the dot product with the line-of-sight vector must be positive).

[0207] In step 3032, due to the point-to-surface normal orthogonal error The distribution may contain heavy tails and extreme values, and directly using the mean square or standard deviation will result in a few outliers dominating the data. To ensure that subsequent evaluations remain stable for different point cloud densities, scales, and a small number of outliers, robust scaling normalization of these residuals is necessary. Therefore, further calculations are needed regarding the point-to-surface normal orthogonality error. Normalized robust scalar SR and upper tail quantile qu:

[0208] ;

[0209] ;

[0210] in, To obtain the median function, It is a very small positive number to prevent errors in subsequent calculations of quantile qu. Dividing by zero, in some embodiments, may be set ; To obtain the upper quantile function, λ is the upper quantile truncation ratio. The meaning of the upper tail quantile function is that, across all samples... Let's take a threshold qu such that approximately 1-λ samples do not exceed it, while approximately λ samples exceed it.

[0211] In step 3033, the point cloud geometric verification similarity metric for determining whether the point cloud geometric verification passes is finally constructed using the quantile qu.

[0212] ;

[0213] Where ξ is the curve steepness parameter. Similarity is verified in point cloud geometry. In this model, the parameter ξ adjusts the non-linear steepness of mapping quantiles qu to similarity. When ξ is larger, the curve drops faster as qu increases, making it more sensitive to larger residuals and significantly widening the boundary between true and false samples. However, it is also more susceptible to score fluctuations caused by noise or slight jitter. When ξ is smaller, the curve is flatter, and the scoring is more stable, but the discriminative power of marginal samples decreases. Its sensitivity meets the following requirements. This indicates that increasing ξ under the same qu will amplify the absolute value of the slope. In practical applications, after fixing λ on the validation set, the algorithm can iterate through the... The value of ξ within the range is selected based on the highest area under the curve (AUC, the area enclosed by the ROC curve and the coordinate axis) or the lowest error rate (EER). For low noise and strong discrimination ability requirements, ξ = 2 to 3; when the requirement is to ensure stable discrimination ability and low discrimination error (but the noise is relatively large), ξ is generally 1.5.

[0214] Point cloud geometric similarity criteria The closer the value is to 1, the smaller the error in the normal direction for most points in the two point clouds, and the better the fit. In practical use, simply... By comparing the results with the set second upper threshold and second lower threshold, it can be directly determined whether the geometric verification is passed, reviewed, or rejected.

[0215] Geometric verification of point cloud similarity The results are compared with a second upper threshold and a second lower threshold to determine whether the point cloud geometric verification is passed. Through training statistics, the second upper threshold is... Set to 0.72, second lower threshold. Set to 0.42, if If the output point cloud geometry verification passes, then the face recognition passes; if If the point cloud geometry verification fails, an alarm will be triggered, an auxiliary authentication process will be initiated, or a rejection result will be output. If the output is uncertain, point cloud re-verification is required. After the number of point cloud re-verification attempts is exhausted, a rejection result will be output, or an alarm will be triggered or an auxiliary identity verification process will be entered.

[0216] Point cloud re-verification can be achieved by repeatedly acquiring the same face region within a short time window or extracting sub-point clouds from consecutive frames. After preprocessing and registration, the geometric similarity of each frame is robustly fused (frames with more internal points and better registration convergence are preferred). If multiple frames consistently reach the second upper threshold, the result is considered successful; otherwise, it is considered a failure or uncertain.

[0217] Figure 8 is a schematic diagram of the processing results of each sub-step of the point cloud geometric verification method of this application. It shows the geometric verification similarity comparison results of two sets of sample point clouds after voxel downsampling, outlier removal, normal estimation, FPFH local descriptor extraction, weighted SVD robust registration and scale normalized residual statistics.

[0218] The first row of Figure 8 shows the processing results of each step in the point cloud geometric verification of the RGB image verification results in the first row of Figure 3. The second row shows the processing results of each step in the point cloud geometric verification of the RGB image verification results in the second row of Figure 3. Combining the sub-results of each step in the point cloud geometric verification of the two rows (corresponding to typical examples of point cloud geometric verification failure and success respectively), in the voxel raster downsampling process in step 3011, compared with the original dense point cloud of the original face image in the first row and first column of Figure 3, voxel downsampling replaces the points within the same voxel with the voxel's center point, making the overall point cloud density more uniform and significantly reducing redundant points, while the facial contour remains intact. This change directly reduces the computational load of subsequent descriptor calculation and matching, and suppresses the correspondence bias caused by local over-dense sampling, thereby establishing a comparable geometric representation basis for the two clouds under different sampling conditions. Subsequently, in the sub-image of the processing result in step 3012 (outlier extraction result image), compared with the voxel raster downsampling result image processed in step 3011, it can be seen that background floating points, edge burrs, and scattered noise have been significantly removed, and the facial subject point set is more concentrated and the contour is cleaner. This step prevents the candidate correspondence from being pulled by a small number of outliers, reducing the probability of "erroneous in-points" or local collapse in the robust registration stage, and ensuring that subsequent registration is mainly based on the real facial geometry. Further, in the normal result image processed in step 3013, compared with the pure position point cloud of the outlier extraction result image, the point cloud is given a local normal direction (visualized by colored normals in the image), thus providing a stable tangent plane reference for subsequent point-to-surface residuals and consistency constraints. This change means that subsequent error measurement no longer depends on point-to-point distance, but is evaluated around the "normal orthogonal component", fundamentally reducing the impact of tangential misalignment, uneven sampling, and local holes on error amplification, making the geometric consistency evaluation more consistent with the real surface fit.

[0219] It should be noted that, in order to protect the portrait rights of individuals in the original face recognition image, the processing results of each step of the RGB image verification in Figure 3 and the processing results of each step of the point cloud geometry verification in Figure 8 of this application have been blurred.

[0220] After completing the preprocessing in step 301, the process continues in step 302. The FPFH local descriptor result after step 3021 further introduces local geometric descriptor representations compared to the sub-graph of the normal estimation result (the feature response or the local feature distribution after encoding is presented as colored discrete points in the figure). The point cloud after this step is no longer just a set of shape points, but carries comparable local structural signatures. The beneficial effect of this is that it can establish more reliable candidate correspondences in stable geometric regions such as the bridge of the nose, eye socket, and cheekbone, providing sufficient "verifiable matching clues" for subsequent robust estimation. Subsequently, after processing in step 3023, compared to the state with only local descriptors and no alignment, after robust filtering and solving for the optimal rigid body transformation parameters, the two point clouds tend to be consistent in overall pose, and the overlap relationship of key facial structures is significantly enhanced. By eliminating erroneous matches through the consistency constraint in step 3023, only the set of interior points is retained to support the rigid body solution, so that the alignment result is more determined by stable facial geometry, rather than by noise points, occlusion points or accidental matches. Therefore, even when there are local missing points and depth jitter, an interpretable and reproducible alignment effect can still be obtained. Finally, in the sub-image processed in step 303, compared to the sub-image processed in step 302 where only the alignment results can be observed, this sub-image further normalizes and statistically aggregates the aligned normal residuals, and visually presents the residual distribution and final similarity values ​​using heatmap color scales. This effectively transforms feature alignment and alignment accuracy into quantifiable, threshold-manageable point cloud geometric verification similarity. Furthermore, robust scaling and quantile statistics suppress the influence of heavy-tailed and extreme outliers, ensuring the judgment is not influenced by a few abnormal regions. Ultimately, since the Spc result of the original image corresponding to the LBPH method after point cloud geometric verification is 0.963, which is greater than the second upper threshold, the face passes the face recognition of this application, and the verification result is output as passed. The Spc result of the original image from which the face descriptor is extracted by the CNN method after point cloud geometric verification is approximately 0.396, which is less than the second lower threshold, therefore a rejection result is output.

[0221] By comparing the point cloud geometric verification results of the face descriptor extraction by the LBPH method and the point cloud geometric verification results of the face descriptor extraction by the CNN method in the first and second rows of Figure 8, respectively, it can be seen that this application processes the point cloud geometric consistency through the normal orthogonal residual in step 301, the robust normalization in step 302, and the statistical convergence in step 303, thereby transforming the point cloud geometric consistency into a stable judgment index. Thus, even when the RGB appearance is similar or even when the RGB re-verification passes, it can still effectively identify cases where the three-dimensional geometry is inconsistent and the person is not the same, reduce the false pass rate, and improve the adaptability to noise and sampling differences, so as to give more stable judgment face recognition results in real-world scenarios.

[0222] The technical solution of this application can be implemented by program instructions carried on a computer-readable storage medium. When the instructions are executed by a processor, the computer or electronic device completes a sequential verification process based on RGB and point clouds, including steps such as image preprocessing, descriptor extraction, similarity and confidence calibration, point cloud preprocessing, robust registration, and geometric similarity calculation and classification. The computer-readable storage medium can be a non-transitory medium, such as a read-only memory, random access memory, flash memory, solid-state drive, disk, optical disk, memory card, or any combination thereof; the program can be deployed in the form of independent software, firmware, containerized service, or driver, and can be loaded and updated locally or over a network.

[0223] The electronic device may include, but is not limited to, servers, edge computing gateways, industrial controllers, embedded AI terminals, laptops, mobile terminals, access control all-in-one machines, or camera modules, and at least includes a processor, memory, communication interface, and electrically connected RGB imaging sensors and point cloud sensors. The point cloud acquisition component can be any of the following: structured light, time-of-flight depth camera, binocular stereo camera, or small LiDAR. The processor can be a general-purpose CPU, graphics processing unit, or neural network accelerator. When the processor executes the program instructions, the device completes face detection and cropping, image equalization and illumination compensation, LBPH or convolutional neural network feature extraction, feature comparison and threshold determination, voxel downsampling, outlier removal, normal estimation, robust registration, and geometric similarity calculation, and then outputs pass, review, or rejection results, and, when necessary, connects to password, fingerprint, or manual review for fallback verification.

[0224] It should be understood that the above software and hardware implementations can be replaced or tailored according to the deployment scenario. The functional modules can be implemented as one or more processes, threads, or microservices, and can be executed sequentially on the same processor or in parallel on multiple processors. The order of the steps is not uniquely limited, and equivalent substitution and reorganization are allowed provided that the logic and functionality are satisfied. Where there is no conflict, the technical features described in the embodiments of this application can be combined arbitrarily, and all such combinations fall within the protection scope of this application.

Claims

1. A face recognition method based on sequential verification of point cloud data and RGB images, characterized in that, The process includes the following steps: acquiring RGB image data and point cloud data corresponding to the same moment; performing face detection, cropping, and feature extraction on the RGB image data to obtain RGB face features, and comparing them with a face feature database to obtain RGB verification similarity, thereby obtaining the RGB image verification result; Based on the RGB image verification result, the system selects to proceed to point cloud geometric verification or RGB image re-verification. When entering point cloud geometric verification, geometric feature extraction and matching operations are performed on the point cloud data corresponding to the RGB face region to obtain the point cloud geometric verification similarity, thereby obtaining the point cloud geometric verification result. Based on the point cloud geometric verification result, a face recognition identity determination result is made or point cloud geometric re-verification is performed. Only when the point cloud geometric verification meets the passing conditions is the face recognition verification passed, at which point the identity result of the identified object and the corresponding confidence information are output; otherwise, a face recognition failure or uncertain result is output. When RGB image re-verification or point cloud geometric re-verification fails, the current recognition result is submitted to the warning and fallback verification system to trigger at least one of password verification, fingerprint verification, or manual review to complete the identity confirmation.

2. The method according to claim 1, characterized in that, The RGB image verification adopts a hierarchical judgment mechanism, including: when the RGB verification similarity is higher than a first upper threshold, point cloud geometric verification is directly triggered; when the RGB verification similarity is lower than a first lower threshold, RGB verification is determined to be unsuccessful and the current recognition process is terminated; when the RGB verification similarity is between the first upper threshold and the first lower threshold, it is marked as a blurred state and enters the RGB re-verification process; the RGB re-verification process includes: re-acquiring RGB image data for the same target at different time frames or under different sampling conditions and re-calculating the RGB verification similarity; statistically fusing the multiple RGB verification similarity results; when the fusion result is higher than the first upper threshold and each result is not lower than the first lower threshold, the point cloud geometric verification stage is allowed, otherwise an alarm is triggered or the auxiliary identity verification process is entered.

3. The method according to claim 1, characterized in that, The point cloud geometric verification also adopts a hierarchical judgment mechanism, including: when the point cloud geometric verification similarity is higher than the second upper threshold, the point cloud verification is judged to pass; when the point cloud geometric verification similarity is lower than the second lower threshold, the point cloud verification is judged to fail; when the point cloud geometric verification similarity is between the second upper threshold and the second lower threshold, the point cloud re-verification process is entered; the point cloud re-verification process includes: re-acquiring the point cloud data of the target corresponding region or extracting the face point cloud set from multiple consecutive frames of point cloud data and performing the point cloud geometric verification again; calculating the point cloud geometric verification similarity for each point cloud set and performing set-level fusion judgment; when the set-level point cloud geometric verification similarity is higher than the second upper threshold within a specified number of times, the point cloud verification is judged to pass; when the set-level point cloud geometric verification similarity is lower than the second lower threshold within a specified number of times, an alarm is triggered or an auxiliary identity verification process is entered or a rejection result is output; other cases are judged as uncertain and point cloud re-verification continues or an alarm is triggered or an auxiliary identity verification process is entered after the number of attempts is exhausted.

4. The method according to claim 1, characterized in that, The steps of the RGB image verification include: performing face detection on the input RGB face image to obtain face bounding boxes; cropping and scaling the face region to a preset size based on the face bounding boxes; and performing image equalization and illumination compensation processing on the cropped image to obtain a preprocessed face image; extracting face descriptors based on the preprocessed face image, wherein the face descriptor extraction method is either the LBPH method or the CNN method, and normalizing the extracted face descriptors to obtain unit feature vectors; calculating the similarity between the unit feature vectors and reference feature vectors in the face feature library, and calibrating the confidence level of the similarity to obtain an RGB verification similarity; and outputting the RGB image verification result based on the comparison results of the RGB verification similarity with a first upper threshold and a first lower threshold, and triggering subsequent sequential verification processes.

5. The method according to claim 4, characterized in that, The step of obtaining the preprocessed face image includes: detecting face bounding boxes for each input RGB face image using a face detector, and cropping the face region based on the face bounding boxes and scaling it to a fixed size image with a preset side length; performing image equalization processing on the cropped and scaled face image to obtain an equalized image, wherein, when using LBPH to extract face descriptors, the cropped and scaled image is first converted to grayscale before performing histogram equalization or contrast-limited adaptive histogram equalization processing; and performing illumination compensation on the equalized image to obtain an illumination-compensated image, wherein the illumination compensation includes performing nonlinear correction of pixel intensity based on gamma coefficients, and when the contrast-limited adaptive histogram equalization processing reaches a preset enhancement intensity, the illumination compensation step can be omitted to avoid over-enhancement.

6. The method according to claim 4, characterized in that, The steps of extracting face descriptors and obtaining unit feature vectors based on the preprocessed face image include: when using the LBPH method, calculating the Local Binary Pattern (LBP) encoding pixel-by-pixel on the preprocessed face image with a preset radius and a preset number of neighborhood samples; dividing the face image into a preset number of sub-grids, calculating the LBP encoding histogram for each sub-grid and performing normalization processing; concatenating the normalized histograms of each sub-grid in sequence to obtain the LBPH feature vector, and performing L2 normalization processing on the LBPH feature vector to obtain the unit feature vector; when using the CNN method, the steps include: performing pixel normalization processing on the preprocessed face image, normalizing the pixels based on the channel mean and channel standard deviation statistically obtained from the training set to obtain the network input tensor; inputting the network input tensor into the trained convolutional neural network for forward inference to obtain the embedded feature vector, and performing L2 normalization processing on the embedded feature vector to obtain the unit feature vector; wherein, the unit feature vector serves as the input for subsequent similarity calculation, enabling different descriptor extraction methods to be uniformly compared under the same threshold judgment framework.

7. The method according to claim 1, characterized in that, The point cloud geometric verification includes: performing voxel downsampling, outlier removal, and normal estimation preprocessing on the source and reference point clouds to ensure that the two sets of point clouds meet the stability requirements for subsequent matching and consistency evaluation in terms of point density, noise, and local geometric representation; calculating local geometric descriptors for the preprocessed point clouds and establishing point-to-point correspondences; selecting interior point sets that meet error constraints from candidate correspondences using an iterative robust estimation method; minimizing the registration residuals based on the interior point sets to obtain the optimal rigid body transformation parameters to achieve robust registration from the source point cloud to the reference point cloud; calculating the normal orthogonal residuals based on the optimal rigid body transformation parameters; performing robust normalization and statistical aggregation on the residuals to obtain the point cloud geometric verification similarity; comparing the point cloud geometric verification similarity with a second upper threshold and a second lower threshold, and outputting a pass / fail, review, or rejection result.

8. The method according to claim 7, characterized in that, The preprocessing steps for the source and reference point clouds include: performing voxel grid downsampling on the source and reference point clouds using a preset voxel edge length, replacing points falling within the same voxel with their centroids to obtain a set of voxel centroids after downsampling; for the point cloud after voxel grid downsampling, calculating the average Euclidean distance from each point to its neighboring points within a preset number of neighboring points, and identifying and removing points whose average Euclidean distance exceeds a threshold determined by the global mean and standard deviation; for the point cloud after removing outliers, constructing a local three-dimensional spherical neighborhood with a preset neighborhood radius or a preset number of neighboring points, forming a covariance matrix based on the set of neighboring points and performing eigenvalue decomposition, and taking the eigenvector corresponding to the smallest eigenvalue as the estimated normal vector of the point.

9. The method according to claim 7, characterized in that, The steps for identifying and removing outliers include: after preprocessing, calculating local descriptors for both the source point cloud and the reference point cloud within a preset statistical neighborhood, and normalizing the local descriptors to obtain local feature vectors; based on the local feature vectors, determining the corresponding point in the reference point cloud with the smallest descriptor distance for each source point in the source point cloud, and establishing a candidate matching pair set between the source point and the corresponding point; performing robust consistency screening on the candidate matching pair set to retain inlier matching pairs that satisfy the local descriptor pair retention threshold constraint; constructing a residual minimization model based on the inlier matching pair set and solving for the optimal rigid body transformation parameters from the source point cloud to the reference point cloud.

10. The method according to claim 7, characterized in that, The steps for calculating the geometric verification similarity of point clouds include: based on the obtained optimal rigid body transformation parameters, performing a rigid body transformation on the source points in the source point cloud and determining their corresponding points in the reference point cloud; using the estimated normal vector of the corresponding points to calculate the normal orthogonal residual of the transformed source points relative to the local tangent plane of the corresponding reference points; performing robust scaling estimation and scaling normalization on the normal orthogonal residuals, and extracting upper quantile statistics from the normalized residual set to suppress the influence of heavy-tailed distributions and extreme outliers on the statistical results; constructing a point cloud geometric verification similarity index by monotonically nonlinear mapping the upper quantile statistics; and comparing the point cloud geometric verification similarity index with a second upper threshold and a second lower threshold to determine whether the point cloud geometric verification is passed, enters re-verification, or directly outputs a rejection result.

Images