Biological feature scanning and identification system for border inspection fast clearance

Abstract

This invention discloses a biometric scanning and recognition system for rapid border inspection clearance. Within a single acquisition event, it acquires short facial and iris sequences, binding each frame with a unified hardware clock for exposure center time, exposure time, line exposure center delay mapping, and camera calibration parameters. It performs detection and localization, mask generation, and pose estimation on the facial and iris frames. Based on optical flow and exposure timing, it generates corrected images and motion degradation amounts. It obtains a cross-modal pose-consistent effective texture index based on effective facial quality, effective iris texture quality, and pose mismatch penalty, selecting the unique frame pair with the largest index. Based on this frame pair, it extracts facial embedding, iris binary codes, and effective bit masks, and modulates the cross-modal fusion distance using the index, outputting identity recognition results and a recognized image.

Description

Technical Field

[0001] This invention belongs to the field of identity verification technology for border inspection channels, specifically relating to a biometric scanning and recognition system for rapid border inspection clearance. Background Technology

[0002] Self-service border inspection channels are typically equipped with visible light facial cameras, near-infrared iris cameras, trigger controllers, supplementary lighting units, image acquisition cards, local processors, and registration template libraries. The equipment needs to complete image acquisition, localization, normalization, template comparison, and channel result output within a short timeframe as passengers pass through the acquisition area. In current engineering implementations, the facial pathway typically focuses on localization and embedding extraction under a wide field of view, while the iris pathway typically focuses on eye localization, ring unwrapping, and code stream comparison under near-infrared illumination. The two results are then weighted by scores or fused according to rules to generate an identity verification result. Publicly available solutions such as CN1932840A have addressed joint iris and face acquisition and fusion recognition, US20160055387A1 has addressed the association of facial and iris images in sequences, and CN102844766B has addressed the fusion of multiple information from eye images.

[0003] In actual deployments at border inspection channels, the field of view, focal length, exposure window, illumination method, and frame rate of facial and iris cameras often differ. Single-sensor devices also introduce sequential exposure intervals when switching between visible and near-infrared illumination. When passengers move naturally within the data collection area, their head orientation, eyelid state, eye optical axis, and facial anchor point position may change within a short period. If the system only checks whether the quality of a single image meets a threshold before fusing facial and iris frames separately, it may send two inputs from different short-term states into the same template comparison process. This approach underutilizes hardware clock, line exposure delay, camera calibration, and cross-field projection deviation, making the fusion distance susceptible to the asynchronous nature of the two inputs, thus increasing false rejections, re-collection, or channel waiting. For border inspection channel equipment requiring continuous operation, it is also necessary to retain standardized facial images, iris stripes, effective mask contours, and data collection process records while outputting identity recognition results to support equipment-side verification and operation maintenance. Summary of the Invention

[0004] The present invention aims to provide a biometric scanning and recognition system for rapid border inspection clearance, which solves the technical problems mentioned in the background art.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution.

[0006] A biometric scanning and recognition system for expedited border control, comprising acquiring short facial and iris sequences within a single data acquisition event, configured to execute: The exposure center time is determined for each frame in the facial short sequence and the iris short sequence based on a unified hardware clock. A candidate face frame set and a candidate iris frame set are formed from the short facial sequence and the short iris sequence, respectively. The cross-modal pose-consistent effective texture index of candidate frame pairs is calculated based on the effective facial quality of candidate face frames, the effective iris texture quality of candidate iris frames, and the pose mismatch penalty between candidate face frames and candidate iris frames. A unique frame pair is determined based on the cross-modal pose-consistent effective texture index; Based on the unique frame pair, standardized feature extraction and fusion matching are performed, and the cross-modal fusion distance is modulated according to the cross-modal pose-consistent effective texture index corresponding to the unique frame pair. The identity recognition result is output according to the cross-modal fusion distance.

[0007] Compared with the prior art, the present invention has the following substantial features and significant progress: Compared to the method of selecting frames for facial and iris pathways separately based on single-path scoring and then performing fixed-weight fusion, this invention binds the same hardware clock and camera calibration parameters to the two types of image frames at the channel acquisition end. At the processing end, it uses motion degradation, visible iris texture, exposure time difference, pose difference, and cross-field projection difference to jointly screen unique frame pairs. At the template comparison end, it uses the cross-modal pose-consistent effective texture index corresponding to the frame pair to modulate the fusion distance. This ensures that the facial, iris, and mask inputs entering the matching process have a traceable source from the same acquisition, which helps reduce false rejections, re-acquisitions, and channel waiting caused by the two images not being in the same short-term state. At the same time, it preserves the implementable carriers such as cameras, sensors, hardware clocks, acquisition buffers, template libraries, and recognition image outputs in the border inspection equipment. Attached Figure Description

[0008] Figure 1 This is a module architecture diagram of the biometric scanning and recognition system for rapid border inspection of the present invention. Detailed Implementation

[0009] The following description is provided in conjunction with the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the invention; those skilled in the art can make equivalent substitutions or combinations for the specific implementations.

[0010] The following combination Figure 1 Specific embodiments of the present invention will be described below. Figure 1 The system architecture and processing mechanism diagram of the biometric scanning and identification method used for rapid border control are shown. Figure 1In this system, a unified hardware clock 110 provides a unified time reference for the facial visible light camera 120, the iris near-infrared camera 130, and the single-sensor sequential exposure device 140; the camera calibration parameter storage unit 150 provides camera calibration parameters and line exposure center delay mapping to the acquisition buffer 160 and the detection, positioning, and mask generation module 200; the acquisition buffer 160 sends the facial short sequence and iris short sequence bound with exposure timing information and camera calibration parameters to the processor 100; the processor 100 is equipped with a detection, positioning, and mask generation module 200, a motion degradation correction module 210, a cross-modal index calculation module 220, a unique frame pair selection module 230, a standardized feature extraction module 240, a fusion matching module 250, and a result image generation module 260; the registration and clearance template library 300 provides the fusion matching module 250 with template facial embedding vectors, template iris binary codes, and template effective bit masks; the result image generation module 260 outputs a border inspection rapid clearance biometric recognition image 400.

[0011] In this embodiment, the processor 100 can be a central processing unit, image processor, vision processing chip, edge computing device, or a combination thereof in the border inspection channel terminal. The unified hardware clock 110 can be provided by an image acquisition card, camera trigger controller, or the clock source of the border inspection channel terminal. The facial visible light camera 120 is used to acquire short facial sequences, and the iris near-infrared camera 130 is used to acquire short iris sequences; in another embodiment, the single-sensor sequential exposure device 140 can also acquire facial frames and iris frames in the order of exposure and illumination switching. The camera calibration parameter storage unit 150 can be a non-volatile memory, a configuration file storage area, or the calibration database of the border inspection channel terminal.

[0012] This embodiment processes short facial and iris sequences obtained within a single acquisition event. An acquisition event can be triggered by the optical acquisition area when a passenger enters the border inspection channel, or by a sensor or visual detection at the channel entrance. The acquisition buffer 160 records the frame type, acquisition event number, exposure center time, exposure time, frame interval, line exposure center delay mapping, and camera calibration parameter version for each frame, enabling the processor 100 to enumerate combinations of candidate facial and iris frames within the same acquisition event.

[0013] The main process of this implementation includes: acquiring short facial and iris sequences within a single acquisition event; forming a candidate facial frame set and a candidate iris frame set; calculating the motion degradation, effective facial quality, and effective iris texture quality of the candidate frames; calculating the pose mismatch penalty and cross-modal pose-consistent effective texture index for candidate frame pairs; selecting unique frame pairs according to a deterministic sorting order; performing standardized feature extraction and fusion matching based on the unique frame pairs; modulating the cross-modal fusion distance using the cross-modal pose-consistent effective texture index corresponding to the unique frame pairs; and outputting the identity recognition result and the border inspection rapid clearance biometric recognition image 400 based on the cross-modal fusion distance.

[0014] All modules use the same terminology dictionary and the same data version. Face frames, iris frames, camera calibration parameters, model versions, and registration templates all carry acquisition event numbers or version identifiers. When the processor 100 detects a data version inconsistency, it terminates the fusion matching of that acquisition event and outputs a device configuration anomaly status to prevent data generated by different calibration relationships or different feature extraction models from entering the same calculation.

[0015] Example 1: This embodiment corresponds to Figure 1 The system includes a unified hardware clock 110, a facial visible light camera 120, an iris near-infrared camera 130, a camera calibration parameter storage unit 150, an acquisition cache 160, a processor 100, a registration and clearance template library 300, and a border inspection rapid clearance biometric recognition image 400.

[0016] When a passenger enters the optical acquisition area of ​​the border inspection channel, the unified hardware clock 110 sends a synchronization acquisition trigger to the face visible light camera 120 and the iris near-infrared camera 130. The face visible light camera 120 outputs a short facial sequence, and the iris near-infrared camera 130 outputs a short iris sequence. Each facial frame and each iris frame are written to the acquisition buffer 160 and bound to the same acquisition event number. The acquisition buffer 160 also records the exposure center time, exposure time, frame interval, and line exposure center delay mapping for each frame under the unified hardware clock 110.

[0017] The camera calibration parameter storage unit 150 provides camera intrinsic and extrinsic parameters to the acquisition buffer 160 and the detection, positioning, and mask generation module 200. For a fixed installation scenario with the face visible light camera 120 and the iris near-infrared camera 130, the camera intrinsic parameters include focal length, principal point, and distortion coefficients, while the camera extrinsic parameters include the rotation and translation relationships between the two camera coordinate systems. The processor 100 reads the face short sequence, iris short sequence, exposure timing information, and camera calibration parameters from the acquisition buffer 160 and then executes each processing step.

[0018] The detection, localization, and mask generation module 200 performs detection, localization, mask generation, and pose estimation on facial frames in the short facial sequence, obtaining the face bounding box, inner and outer canthi of both eyes, binocular centers, eye anchor points in the facial image, binocular center distance, occlusion area, and facial head pose vector. The detection, localization, and mask generation module 200 performs detection, localization, mask generation, and pose estimation on iris frames in the short iris sequence, obtaining the pupil boundary, outer iris boundary, eyelid boundary, iris center, area of ​​the complete iris ring, complete iris ring mask, visible iris area, visible iris mask, specular occlusion mask, and eye optical axis pose vector. Frames with complete detection and localization results are respectively added to the candidate facial frame set and the candidate iris frame set.

[0019] The motion degradation correction module 210 obtains the motion degradation amount of each candidate frame based on the optical flow and exposure timing information of adjacent frames in the same short sequence, and generates a corrected facial image and a corrected iris image. The cross-modal index calculation module 220 calculates the cross-modal pose-consistent effective texture index of each candidate frame pair based on the effective facial quality, the effective iris texture quality, and the pose mismatch penalty. The unique frame pair selection module 230 selects a unique frame pair from all candidate frame pairs according to a deterministic sorting order.

[0020] The standardized feature extraction module 240 extracts facial embedding vectors, iris binary codes, and valid position masks based solely on unique frame pairs. The fusion matching module 250 compares the facial embedding vectors, iris binary codes, and valid position masks with template facial embedding vectors, template iris binary codes, and template valid position masks in the registration and clearance template library 300 to obtain the cross-modal fusion distance for each available template. The result image generation module 260 compares the minimum cross-modal fusion distance with an identity determination threshold and outputs the identity recognition result and the border inspection rapid clearance biometric recognition image 400.

[0021] Instead of selecting the face frame with the highest single-modal quality from the short face sequence and the iris frame with the highest single-modal quality from the short iris sequence, processor 100 selects the unique frame pair that ranks first in the cross-modal pose-consistent effective texture index before fusion matching. The data entering the normalized feature extraction module 240 and the fusion matching module 250 come from face frames and iris frames that have temporal, pose, and projection constraints within the same acquisition event.

[0022] In one executable configuration, 8 frames are acquired for both facial and iris short sequences, with an acquisition window of 0.8 seconds. The facial visible light camera 120 acquires at 30 frames per second, and the iris near-infrared camera 130 acquires at 60 frames per second. The processor 100 does not align the two types of short sequences by frame number, but instead uses the exposure center time of each frame to participate in the candidate frame pair calculation. During device deployment, the sequence length and acquisition window can be selected within the range of 3 to 20 frames and 0.2 seconds to 2 seconds. The selection result is fixed on the verification data and then written to the configuration file.

[0023] The trigger lines of the facial visible light camera 120 and the iris near-infrared camera 130 receive the same trigger pulse. The counting resolution of the unified hardware clock 110 is not less than 1 microsecond. When the device starts up, it continuously acquires no less than 100 sets of synchronous trigger frames and calculates the median of the exposure center time difference between the two cameras as a fixed channel delay correction value. During operation, this correction value is first subtracted, and then the exposure center time difference of the candidate frame pair is calculated. When the absolute residual obtained from the synchronization check exceeds 100 microseconds for 10 consecutive sets, the processor 100 outputs a clock synchronization abnormality status.

[0024] Processor 100 performs a defined state transition for each acquisition event. After the acquisition buffer 160 disables writing, it first verifies the frame count, timestamp order, camera calibration parameter version, and pixel array integrity of the facial short sequence and iris short sequence; only after the verification passes is the detection localization and mask generation module 200 started. After the detection localization and mask generation module 200 completes two candidate sets, the motion degradation correction module 210 only reads frames within the set. The cross-modal index calculation module 220 only enumerates the Cartesian product of the two candidate sets and does not access the frames that have been removed. After the unique frame pair selection module 230 generates a unique frame pair, the normalized feature extraction module 240 and the fusion matching module 250 only receive the unique frame pair and its associated parameters and do not reselect single-modal frames.

[0025] Each module associates data by collecting event numbers and frame indices. Facial frame indices are consecutively numbered within short facial sequences, and iris frame indices are consecutively numbered within short iris sequences; frame indices are not compared between the two types of short sequences. Each module's output records the source frame index, processing status, and configuration version. If an output record lacks a source frame index, the processor 100 marks the record as invalid to prevent incorrect association of quality values, pose vectors, or masks with other frames.

[0026] In the dual-camera full execution mode, once the face visible light camera 120 and the iris near-infrared camera 130 are fixedly installed, their focal length, principal point, or relative position cannot be changed during the acquisition event. Cameras with autofocus complete focusing and lock the focus position before the acquisition event begins; when the focus position changes, a new version of the camera calibration parameters is generated. Cameras with automatic exposure can adjust the exposure time frame by frame, but the actual exposure time for each frame must be written to the acquisition buffer 160 to ensure that motion degradation and time mismatch parameters use the true values.

[0027] The processing time limit for a single acquisition event can be configured to 2 seconds. If the processor 100 fails to generate a unique frame pair within the processing time limit, it terminates the acquisition event and outputs a processing timeout status; the processing timeout status is not considered a result of identity recognition. The processing time limit only restricts device response and does not change any quality values, mismatch penalties, or cross-modal fusion distance. This separation method prevents business time limits from being incorporated into biometric calculations.

[0028] The result image generation module 260 receives a standardized facial image, a standardized iris strip, an effective bit mask, a frame index for unique frame pairs, cross-modal fusion distance, and identity recognition results. When generating the border inspection rapid clearance biometric recognition image 400, a fixed display size is used for the facial image and iris strip, and no visual reconstruction of the facial embedding vector, iris binary code, or registration template is performed. The output image is only associated with the acquisition event number and no additional rules that change the identity recognition calculation are written.

[0029] Example 2: This embodiment corresponds to Figure 1 The unified hardware clock 110, face visible light camera 120, iris near-infrared camera 130, single sensor sequential exposure device 140, camera calibration parameter storage unit 150 and acquisition buffer 160 are used to illustrate the binding method of exposure timing information and camera calibration parameters.

[0030] In the dual-camera acquisition mode, a unified hardware clock 110 provides the same hardware monotonic clock reference to both the face visible light camera 120 and the iris near-infrared camera 130. When the face visible light camera 120 outputs a face frame, it writes the exposure center time, exposure time, and frame interval into the frame metadata; when the iris near-infrared camera 130 outputs an iris frame, it also writes the exposure center time, exposure time, and frame interval into the frame metadata. The acquisition buffer 160 archives the face short sequence and iris short sequence according to the acquisition event number, and saves the frame metadata along with the image pixel array.

[0031] In the single-sensor acquisition method, the single-sensor sequential exposure device 140 acquires facial and iris frames according to a fixed exposure and illumination switching sequence. In one acquisition cycle, a facial frame is acquired first under visible light illumination, then the visible light illumination is turned off and the near-infrared illumination is turned on to acquire an iris frame. The corresponding exposure is only initiated after the illumination switching completion signal arrives. The acquisition buffer 160 retains the actual exposure center time of each frame, enabling the cross-modal index calculation module 220 to generate a pose mismatch penalty based on the exposure center time difference.

[0032] For a face visible light camera 120, an iris near-infrared camera 130, or a single-sensor sequential exposure device 140 employing rolling shutter exposure, the camera calibration parameter storage unit 150 provides a line exposure center delay map to the acquisition buffer 160. The line exposure center delay map describes the time offset of each line within the same image frame relative to the frame's exposure center. For global exposure devices, the line exposure center delay map is set to zero, and the processor 100 still uses the same data field.

[0033] Camera calibration parameter storage unit 150 stores camera intrinsic and extrinsic parameters. The camera intrinsic parameters are used for image distortion correction and projection relationship construction; the camera extrinsic parameters are used to establish the projection relationship from the iris field of view to the face field of view. Acquisition buffer 160 associates the camera calibration parameter version with the acquisition event number, ensuring that all frames within an acquisition event use the same version of the camera calibration parameters.

[0034] Each frame in the acquisition buffer 160 includes at least a frame type field, acquisition event number, exposure center time, exposure time, frame interval, line exposure center delay mapping, camera intrinsic parameter identifier, camera extrinsic parameter identifier, camera calibration parameter version, and pixel array address. The frame rates of the face short sequence and the iris short sequence can be different, and the processor 100 directly uses the exposure center time when enumerating candidate frame pairs.

[0035] When the facial short sequence and iris short sequence reach their configured frame counts, or the acquisition window reaches its configured time, or the passenger leaves the optical acquisition area, the acquisition buffer 160 closes the current acquisition event and sends the facial short sequence and iris short sequence within the same acquisition event to the processor 100. The acquisition buffer 160 prohibits combining frames with different acquisition event numbers into candidate frame pairs.

[0036] The exposure center time is calculated as the sum of the exposure start hardware count value and half of the exposure duration count value output by the camera. When the camera only outputs the exposure end hardware count value, the exposure start hardware count value is first obtained by subtracting the exposure duration count value from the exposure end hardware count value, and then the exposure center time is calculated in the same way. The processor 100 converts the hardware count value into the count value of the unified hardware clock 110 and writes it into the acquisition buffer 160.

[0037] The line exposure center delay map is generated during the equipment calibration phase. For those with... The image sensor reads the camera's line period configuration value. , No. The row offset is written using the following formula: An array, where row indices start from 0: ; When the camera cannot provide a line period configuration value, take no fewer than 20 sets of images using a pulsed light source at different trigger delays, and fit the line position based on the brightness-dark boundary. The line exposure center delay map is saved when the fitting residual does not exceed one hardware clock count cycle; a camera calibration anomaly is marked when the fitting residual exceeds this value. The mapping array is stored together with the camera calibration parameter version.

[0038] The acquisition buffer (160) stores frames of the same type in ascending order of exposure center time. When a duplicate frame number is detected, the frame with a valid hardware verification and complete pixel array is retained; when timestamp rollback, invalid pixel array address, or inconsistent camera calibration parameter versions are detected, the corresponding frame is discarded. When the acquisition event is closed, if any short sequence contains fewer than 3 frames, an acquisition failure status is output.

[0039] Frame metadata is stored sequentially using fixed fields. The frame type field takes either a face frame or an iris frame; the acquisition event number uses a monotonically increasing integer; the exposure center time, exposure time, and frame interval use the count value of a unified hardware clock 110; the row exposure center delay mapping uses an array accessed by row index; and the camera intrinsic parameter identifier, camera extrinsic parameter identifier, and camera calibration parameter version use integers. The pixel array address is recorded along with the image width, image height, number of channels, and row stride so that the processor 100 can verify the pixel array boundaries.

[0040] When the number of acquired events reaches the storage limit, processor 100 first clears unused cached records and then generates new acquired event numbers from zero; old records with the same number must not be retained before the numbering restarts. When the unified hardware clock 110 wraps around, the acquisition cache 160 uses the wrap-around count and the current count to form an extended count value. The extended count value must continue to increase monotonically; otherwise, the current acquired event is disabled and a clock abnormality status is output.

[0041] The frame interval is obtained by the difference in exposure center times between adjacent frames in the same short sequence. The acquisition buffer 160 retains the original frame interval field output by the camera for verification, but the motion degradation correction module 210 uses the difference in exposure center times. When the difference between the two exceeds five percent of the original frame interval, the latter of the two adjacent frames is marked as having abnormal timing information.

[0042] A preferred acquisition sequence for the single-sensor sequential exposure device 140 is a cycle of face frame, iris frame, face frame, and iris frame. Both visible light illumination off confirmation signals and near-infrared illumination on confirmation signals are written to the device log. The illumination stabilization time is preferably between 0.5 milliseconds and 2 milliseconds. The exposure start time must not be earlier than the illumination stabilization completion time. If the illumination confirmation signal is missing or both illuminations are simultaneously on, the current frame is not written to the acquisition buffer 160.

[0043] The exposure time of the facial visible light camera 120 is preferably between 0.5 ms and 8 ms, and the exposure time of the iris near-infrared camera 130 is preferably between 0.1 ms and 4 ms. The device selects an exposure time range that simultaneously satisfies the corresponding threshold for both exposure quality and the proportion of visible iris area based on the verification data. If the actual exposure time exceeds the fixed range during operation, the current frame can be retained in the original log but will not be included in the candidate set.

[0044] The line exposure center delay mapping strictly corresponds to the image height. When the camera changes the cropping area or pixel merging mode, the processor 100 rereads the image height and line period configuration value and generates a new mapping array. If the length of the mapping array is inconsistent with the image height, there are missing values ​​in the array, or the offset direction of adjacent lines is inconsistent with the camera readout direction, a camera calibration abnormality status is output.

[0045] Example 3: This embodiment corresponds to Figure 1 The camera calibration parameter storage unit 150, the detection and localization and mask generation module 200, and the cross-modal index calculation module 220 are used to illustrate detection and localization, mask generation, and attitude estimation.

[0046] The detection, localization, and mask generation module 200 receives short facial and iris sequences output from the acquisition buffer 160. For each facial frame, a face bounding box is first detected, and then the inner and outer corners of the eyes, the centers of both eyes, and the eye anchor points in the facial image are located within the face bounding box. The binocular center distance is obtained from the pixel distance between the two eye centers and is used as the ROI pixel feature scale for motion degradation, projection difference normalization, and normalized feature extraction. The facial head pose vector is obtained from facial keypoints, camera intrinsics, and a 3D standard face point set.

[0047] For each iris frame, the detection, localization, and mask generation module 200 detects the pupil boundary, outer iris boundary, and eyelid boundary, and obtains the iris center and the area of ​​the complete iris ring accordingly. The complete iris ring mask is formed by the pixels between the pupil boundary and the outer iris boundary. The visible iris mask is formed by subtracting eyelid occlusion, highlight occlusion, and unusable areas from the complete iris ring mask; the visible iris area is the number of effective pixels in the visible iris mask. The eye optical axis pose vector is obtained from the pupil center, iris center, eye contour, and camera calibration parameters.

[0048] The detection, positioning, and mask generation module 200 establishes the projection relationship from the iris field of view to the face field of view based on the camera calibration parameters provided by the camera calibration parameter storage unit 150. This projection relationship is denoted as... In this embodiment, a local planar homography matrix of the eye is used as the executable projection method. The homography matrix is ​​obtained by acquiring the same calibration target from a fixedly installed facial visible light camera 120 and an iris near-infrared camera 130.

[0049] The detection, localization, and mask generation module 200 outputs biometric ROI, facial key points, iris boundaries, complete iris ring mask, visible iris mask, and specular occlusion mask to the motion degradation correction module 210. The detection, localization, and mask generation module 200 also outputs the facial head pose vector, eye optical axis pose vector, and the projected positions of the facial image eye anchor points and iris center mapped to the facial field of view via projection relationships to the cross-modal index calculation module 220.

[0050] When the detection, localization, and mask generation module 200 fails to obtain the face bounding box, eye center, or eye anchor point in a certain face frame, that face frame is not included in the candidate face frame set. Similarly, when the detection, localization, and mask generation module 200 fails to obtain the pupil boundary, outer iris boundary, visible iris mask, or iris center in a certain iris frame, that iris frame is not included in the candidate iris frame set.

[0051] The facial head pose vector and the eye optical axis pose vector are transformed into a comparable vector space before entering the pose mismatch penalty calculation. In this embodiment, both are represented as three-dimensional unit direction vectors in the 120 coordinate system of the facial visible light camera.

[0052] Facial landmark detection uses an input size of The keypoint detection model outputs pixel coordinates and confidence scores for stable points at the inner and outer corners of both eyes, the tip of the nose, the alar of the nose, the corners of the mouth, and the jawline. The model output coordinates are mapped back to the face frame according to the scaling and offset of the face bounding box in the original image. The face detection and localization results are complete when the confidence scores of the four eye corner keypoints are all not less than 0.5 and the proportion of the face bounding box's area to the intersection of the face bounding box and the image region is not less than 0.8.

[0053] The facial head pose vector was solved using PnP. A one-to-one correspondence was established between the 3D standard face point set and the stable points of the nose tip, nostrils, inner and outer corners of the eyes, corners of the mouth, and jawline. First, the 2D keypoints were corrected based on camera intrinsics and distortion coefficients, and then the rotation matrix was obtained. Taking the third column of the rotation matrix and normalizing it, we obtain the facial head pose vector: ; Iris boundary detection uses grayscale-normalized iris frames as input. First, the segmentation model outputs pixel labels for the pupil region, iris region, eyelid occlusion region, and highlight occlusion region. Then, ellipse fitting is performed on the boundary points of the pupil and iris regions respectively. The ratio of fitted inliers to the pupil and outer iris boundaries is no less than 0.7, and the iris detection localization result is complete when the outer iris boundary surrounds the pupil boundary. The highlight occlusion region is formed by regions with a normalized grayscale value of no less than 0.98 and a connected region area of ​​no less than 4 pixels.

[0054] The eye optical axis pose vector is calculated using the offset of the iris center relative to the pupil center and camera intrinsic parameters. First, the pixel coordinates of the pupil center and iris center are inversely transformed into direction vectors in the camera coordinate system using camera intrinsic parameters. The normalized average of the two direction vectors is then taken as the eye optical axis pose vector. Then use the rotation matrix in the camera extrinsic parameters. Transform to the 120 coordinate system of the visible light camera for the face: ; The local planar homography matrix of the eye is calibrated using at least nine calibration points distributed throughout the local area of ​​the eye. Corresponding points are extracted in the facial and iris fields of view, respectively, and the homography matrix is ​​obtained by direct linear transformation after distortion correction. Then, false matches are eliminated using random sampling consistency. The homogeneous coordinates of the iris center are defined as follows: The projection is calculated as follows: ; when At that time, the output of the projection relationship is: ; After calibration, the reprojection error is calculated using 20 sets of corresponding points that were not used in the solution. The homography matrix is ​​saved when the average reprojection error does not exceed 2 pixels and the maximum reprojection error does not exceed 5 pixels; otherwise, a camera calibration anomaly status is output. When the projected coordinates fall outside the range of the face frame, the corresponding iris frame is not included in the candidate iris frame set.

[0055] The occlusion region is obtained from the pixel classification output of the facial keypoint detection model, marking pixels that cover the effective facial area, such as those from masks, glasses reflections, hat brims, and others. If there is no corresponding pixel classification output, the occlusion region is set to an empty set and undefined regions should not be included in the calculation. The ratio of the number of pixels in the occlusion region to the number of pixels in the biometric ROI is used as the occlusion ratio.

[0056] The face bounding box, the inner and outer corners of both eyes, the centers of both eyes, and the eye anchor points in the facial image all use the original pixel coordinates of the facial frame. The eye anchor points in the facial image are taken as the arithmetic mean of the centers of both eyes. The centers of both eyes are taken as the midpoints of the inner and outer corners of the same eye, respectively. This definition ensures that the eye anchor points in the facial image can be uniquely obtained from the detected inner and outer corners of both eyes and can be directly compared with the facial field of view pixel coordinates output by the projection relationship.

[0057] The 3D standard face point set is fixed in the model version file. The PnP solution uses at least 6 non-collinear 3D points and their corresponding 2D points, and uses iterative reprojection error minimization to obtain the rotation matrix and translation vector. After the solution is completed, the average reprojection error of all participating points is calculated; if the average reprojection error does not exceed 3 pixels and the rotation matrix satisfies orthogonality verification, the face head pose vector is retained; otherwise, the corresponding face frame is not included in the candidate face frame set.

[0058] The iris segmentation model receives a single-channel iris frame. The input image is scaled to 320 pixels along its longer side and zero-padded. The model outputs a pixel classification map with the same size as the input. After the classification map is mapped back to the original iris frame, connected component filtering is performed. Only the largest connected components are retained for the pupil and iris regions, respectively. If either the pupil or iris region is empty, the distance between the centers of two regions exceeds one-quarter of the major axis diameter of the outer iris boundary, or the major axis diameter of the outer iris boundary is less than 40 pixels, the corresponding iris frame is not included in the candidate iris frame set.

[0059] Ellipse fitting of the pupil and outer iris boundaries uses at least 20 boundary points. These boundary points are divided into 36 intervals based on their polar angles, with each interval retaining at most two points with the largest gradient magnitudes to avoid localized dense boundary points dominating the fitting result. The fitted ellipse parameters include the center coordinates, major axis, minor axis, and rotation angle. Pixels within the complete iris ring mask must be located inside the outer iris boundary and outside the pupil boundary.

[0060] The eyelid occlusion region is obtained from the eyelid labels of the segmentation model. The highlight occlusion region is first formed according to the grayscale threshold, and then a dilation process with a radius of 1 pixel is performed to cover the saturated transition region of the highlight edge. The eyelash region is formed by linear regions with a local grayscale value lower than 20% of the median grayscale value of the iris ring and a continuous length of no less than 3 pixels, and is incorporated into the unusable region. It can be seen that the iris mask is obtained by subtracting these regions from the complete iris ring mask.

[0061] The eye optical axis pose vector is calculated using the distortion-corrected pupil center and iris center. The two direction vectors obtained from the inverse transformation of the camera intrinsic parameters are normalized and then their arithmetic mean is taken; the L2 norm of the average vector is not less than [value missing]. Normalization is performed; if the value is less than this, the corresponding iris frame will not be included in the candidate iris frame set. The camera extrinsic parameter rotation matrix undergoes orthogonal verification before use; if the verification fails, a camera calibration anomaly status is output.

[0062] The local planar homography matrix calibration target is set within the spatial range where the eyes are expected to appear in the border inspection channel. Calibration points cover the left and right eye regions in the facial field of view, avoiding the concentration of all points in a single small area. At least five images of the calibration target at different locations are acquired for each calibration, and at least nine corresponding points are extracted from each location. The final homography matrix is ​​solved by combining all corresponding points that have passed the random sampling consistency screening, ensuring that fixed installation errors are covered within the range of eye movement.

[0063] Camera distortion correction is performed before the projection relationship calculation. The iris center is first converted from the original iris frame coordinates to the distortion-corrected iris field of view coordinates, and then homography matrix projection is used; the eye anchor points in the facial image also use the distortion-corrected facial field of view coordinates. After the projection difference calculation is completed, the two coordinates are not converted back to the original distorted image coordinates to prevent coordinate domain mixing.

[0064] Example 4: This embodiment corresponds to Figure 1 The acquisition buffer 160, detection and localization and mask generation module 200, and motion degradation correction module 210 are used to obtain the motion degradation amount of each candidate frame according to optical flow and exposure timing and generate a corrected image.

[0065] The motion degradation correction module 210 receives candidate frames, adjacent frames in the same short sequence, the difference in exposure center time between adjacent frames, biometric ROI, exposure time, ROI pixel feature scale, and row exposure center delay mapping. For face frames, the biometric ROI covers the face bounding box; for iris frames, the biometric ROI covers the entire iris ring.

[0066] For the current frame to be processed Motor degeneration correction module 210 in biometric ROI The optical flow is calculated based on the current frame and the next frame of the same short sequence. And based on the difference in exposure center time between adjacent frames Obtain the average pixel velocity of the ROI: ; in, This represents the median values ​​for the transverse and longitudinal components of the optical flow velocity. This indicates the pixel location within the biometric ROI. When the candidate frame is a face frame, It is the difference in exposure center time between adjacent facial frames; when the candidate frame is an iris frame, It represents the difference in exposure center time between adjacent iris frames.

[0067] Motor degeneration correction module 210 based on exposure time ROI average pixel speed and ROI pixel feature scale Obtain motor degeneration amount: ; in, Used to prevent the denominator from being zero. For face frames, ROI pixel feature scale. Take the center distance between the two eyes; for the iris frame, the ROI pixel feature scale. The major axis diameter of the outer iris boundary is taken. Motion degradation is used to form the effective facial quality and the effective iris texture quality.

[0068] When the candidate frame has a line exposure center delay mapping, the motion degradation correction module 210, according to the first... Line exposure center delay relative to frame exposure center The original image is inversely resampled to obtain the corrected image: ; in, For the original image, To correct the image, and These represent the horizontal and vertical components of the average pixel velocity of the ROI. The origin of the original image coordinate system is located at the top left corner, with positive values ​​for the horizontal direction to the right and the vertical direction to the bottom; the optical flow direction is defined as the direction from the current frame to the next frame. Resampling uses bilinear interpolation. Global exposure device. It is zero.

[0069] The motion degradation correction module 210 outputs the corrected facial image, the corrected iris image, and the corresponding motion degradation amount, respectively. When the effective ratio of optical flow does not meet the motion degradation processing conditions or the number of effective pixels in the biometric ROI is insufficient, the corresponding frame is removed from the candidate set, and the optional branch that combines zeroing and removal is not used.

[0070] Optical flow calculations employ a grayscale image pyramid, with each layer reduced by half, for a total of four layers; each layer uses... Optical flow is calculated within a window. Only optical flow vectors with a forward / backward consistency error of no more than 1 pixel and located within the biometric ROI are retained. The effective optical flow ratio satisfies the motion degradation processing condition when the proportion of effective optical flow to the number of pixels in the biometric ROI is not less than 0.6.

[0071] To limit the applicability of a single translational velocity model, the directional angle and velocity difference between the effective optical flow vector and the average pixel velocity of the ROI are calculated. Resampling is performed on each frame if the effective optical flow vector of at least 0.8 satisfies that the directional angle does not exceed 30 degrees and the velocity difference does not exceed 0.05 times the feature scale of the ROI pixel; otherwise, the frame is removed from the candidate set. This condition prevents rotation, scaling, or opposite-direction motion from being masked by the median result.

[0072] If there is no next frame after the last frame of a short sequence, the reverse optical flow is calculated using the previous frame of the same short sequence, and the negative reverse optical flow is used in the calculation of the average pixel velocity of the ROI. If the short sequence has fewer than 3 frames or the difference between the exposure center times of adjacent frames is not greater than zero, motion degradation processing is not performed and an acquisition failure status is output.

[0073] When the sampling coordinates required for bilinear interpolation fall outside the range of the original image, the corresponding target pixel is marked as an invalid pixel. If the proportion of invalid pixels in the corrected facial image exceeds 0.1, or if the proportion of invalid pixels in the corrected iris image within the complete iris ring exceeds 0.1, the corresponding frame is removed from the candidate set.

[0074] Preferred selection This value ensures that the calculation results at the normal ROI pixel feature scale are largely unaffected by the numerical protection term, and prevents division by zero when the detection scale is abnormally close to zero.

[0075] Before optical flow calculation, the biometric ROIs of the current frame and adjacent frames are expanded to the same rectangular size, and coarse alignment is performed using the detection and localization results. For face frames, translation and coarse scale alignment are performed using the binocular centers and the distance between them; for iris frames, translation and coarse scale alignment are performed using the iris center and the major axis diameter of the outer iris boundary. Coarse alignment is only used to improve optical flow stability; motion degradation still uses the original frame exposure time and the original ROI pixel feature scale.

[0076] The optical flow pyramid is calculated starting from the lowest resolution layer, and the results of each layer are magnified and used as the initial values ​​for adjacent higher resolution layers. The forward optical flow of each pixel is added to the backward optical flow calculated at the corresponding position, and the L2 norm of the resulting vector is used as the forward-backward consistency error. Only pixels with an error of no more than one pixel are included in the median calculation. Occluded, unusable, and specularly occluded regions in biometric ROIs are not included in the optical flow calculation.

[0077] Using the component median instead of the vector arithmetic mean avoids the cancellation of major motion by a small number of anomalous optical flows in opposite directions. For multi-directional motion caused by overall rotation or scaling, this embodiment does not force correction using a single translation model, but rather discards the frame based on the directional angle and velocity difference conditions. Thus, the retained frames meet the applicable conditions for performing row-level translation correction at the average pixel velocity of the ROI.

[0078] The exponential decay of motion degradation is used only for quality calculations and not for directly altering image pixels. The corrected image is generated from the line exposure center delay mapping and the ROI average pixel velocity. Motion degradation is calculated even if the line exposure center delay mapping of the global exposure device is zero, so that motion during exposure can still reduce the effective quality of the face or the effective texture quality of the iris.

[0079] Resampling uses inverse coordinate lookup. This is for correcting image position. The original image sampling coordinates are obtained based on the line exposure center delay and the average pixel velocity of the ROI. Bilinear interpolation is performed when all four adjacent integer pixels of the sampling coordinates are within the image range; any adjacent pixel that exceeds the range is marked as an invalid pixel. Invalid pixels are not included in face quality, iris gradient, feature extraction, or effective bit mask generation.

[0080] The motion degradation correction module 210 outputs a processing status for each candidate frame. The processing status includes successful processing, insufficient effective optical flow ratio, inapplicable motion model, excessive sampling out-of-bounds ratio, and abnormal temporal information. Only frames in the successful processing status enter the cross-modal index calculation module 220. The processing status uses fixed enumeration values ​​and does not determine the calculation branch based on free text.

[0081] In one validation method, short sequences are acquired on a stationary calibration target, a horizontally moving calibration target, and a rotating calibration target. The motion degradation of the stationary calibration target should be close to zero; the motion degradation of the horizontally moving calibration target should increase monotonically with exposure time and movement speed; and the rotating calibration target should be discarded because the motion model is inapplicable. If any of these relationships are not met, the corresponding camera configuration is not enabled.

[0082] Example 5: This embodiment corresponds to Figure 1 The motion degradation correction module 210 and the cross-modal index calculation module 220 are used to describe the effective facial quality and the effective iris texture quality.

[0083] The cross-modal index calculation module 220 receives the corrected facial image, corrected iris image, and motion degradation amount output by the motion degradation correction module 210, and receives the visible iris area, complete iris ring area, complete iris ring mask, visible iris mask, and specular occlusion mask output by the detection localization and mask generation module 200. For candidate facial frames... The effective facial quality is determined based on the normalized facial quality scalar and the motion degradation of candidate facial frames: ; in, To normalize facial quality scalars, This represents the motion degradation of candidate facial frames. The normalized facial quality scalar is calculated based on face bounding box integrity, keypoint confidence, exposure quality, occlusion ratio, and pose availability, and is limited to the range of zero to one.

[0084] For candidate iris frames The cross-modal index calculation module 220 forms the effective texture quality of the iris based on the ratio of the visible iris area to the area of ​​the complete iris ring, the ratio of the gradient magnitude within the visible iris mask to the gradient magnitude within the complete iris ring mask, and the motion degradation of the candidate iris frame. ; in, The visible iris area, The area of ​​the complete iris ring. For visible iris masking, For complete iris ring masking, This is the corrected iris image. The motion degradation amount of the candidate iris frame. and Used to prevent the denominator from being zero.

[0085] The complete iris ring mask is formed based on the pupil boundary and the outer iris boundary. It can be seen that the iris mask is formed by subtracting eyelid occlusion, highlight occlusion, and invalid pixels from the complete iris ring mask. Therefore... for A subset. The grayscale values ​​of the corrected iris image are normalized to the range of zero to one, using... The Scharr operator calculates the lateral and longitudinal gradients, and then calculates the gradient magnitude. The area covered by the specular occlusion mask is not included in the summation of gradient magnitudes within the visible iris mask.

[0086] When the ratio of the visible iris area to the area of ​​the complete iris ring is lower than the visible iris area ratio threshold, the corresponding iris frame is removed from the candidate iris frame set. When the normalized facial quality scalar is lower than the facial quality threshold, the corresponding facial frame is removed from the candidate facial frame set. A removal branch is used, not an arbitrary branch that combines quality zeroing and removal.

[0087] Effective facial quality and effective iris texture quality are used simultaneously to calculate the cross-modal pose-consistent effective texture index and cross-modal fusion distance. Candidate frames are only retained in the candidate set if they meet the conditions for detection integrity, motion degradation processing, and the corresponding quality threshold.

[0088] The normalized facial quality scalar is calculated using the following formula: ; Face frame completeness The keypoint confidence score is calculated by dividing the number of pixels at the intersection of the face bounding box and the image region by the number of pixels in the face bounding box. The arithmetic mean of the confidence scores for stable points at the inner and outer corners of both eyes, the tip of the nose, the ala of the nose, the corners of the mouth, and the jawline; exposure quality. The proportion of pixels within a biometric ROI whose normalized grayscale value is between 0.1 and 0.9; occlusion ratio. The number of pixels in the occluded area is divided by the number of pixels in the biometric ROI.

[0089] Posture availability The yaw angle is obtained from the facial and head posture. Pitch angle and roll angle The angle value is converted to radians before being used in the calculation: ; , , , and All values ​​are limited to the range of zero to one. No calculation is performed if any value is missing. The corresponding facial frames are removed from the candidate facial frame set. The facial quality threshold is preferably 0.4, but can be selected and fixed within the range of 0.3 to 0.5 based on the success rate of facial feature extraction from the validation data.

[0090] It is evident that the preferred threshold for the iris area ratio is 0.3, which can be selected and fixed within the range of 0.25 to 0.4 based on the effective iris code ratio of the verification data. The preferred value is 1. Preferred selection Both the complete iris ring mask and the visible iris mask are stored as binary arrays, with valid pixels set to 1 and invalid pixels set to 0.

[0091] From the range of each sub-item, the mask subset relationship, and the nonnegative motion degradation, we can obtain and This range is used to limit the calculation interval for the cross-modal pose-consistent effective texture index and the cross-modal fusion distance.

[0092] Face bounding box integrity only evaluates whether the face bounding box is cropped by the image boundary, without re-evaluating occluded areas. Keypoint confidence uses the model's original output truncated to zero or one. Exposure quality only evaluates whether pixels are within the usable exposure range, without re-evaluating texture gradients in face quality. Occlusion percentage is obtained solely from the pixel count of the occluded area. Pose usability is obtained solely from the three pose angles.

[0093] The normalized facial quality scalar uses a fixed arithmetic mean and its weights are not learned or updated during runtime. If the device needs to change the importance of the five sub-items, a new configuration version should be generated and the facial quality thresholds recalibrated; within the same configuration version, weights should not be changed based on the passenger or template. This restriction ensures that the same input yields the same result under the same configuration. .

[0094] Before calculating exposure quality, the pixel values ​​of the facial biometric ROI are linearly normalized to the range of zero to one based on the camera's black level and saturation value. The black level and saturation value are stored in the camera calibration parameter storage unit 150. A camera calibration abnormality status is output when the black level is not less than the saturation value. For a three-channel facial image, a single-channel image is first obtained by applying a fixed grayscale conversion coefficient, and then the exposure quality is calculated; the grayscale conversion coefficient remains fixed within the configuration version.

[0095] The iris gradient calculation uses the same normalization method. The Scharr operator outputs the horizontal and vertical gradients, and the gradient magnitude is calculated as the square root of the sum of the squares of the two components. The full iris ring mask and the visible iris mask use the same width and height as the corrected iris image; if the dimensions are inconsistent, the effective iris texture quality is not calculated.

[0096] The number of pixels in the complete iris ring mask is set to avoid different relative effects caused by a fixed constant under different iris image sizes. One is selected to prevent division by zero when the area of ​​the complete iris ring is zero; iris frames with a complete iris ring area of ​​zero have already been eliminated in the detection and positioning stage, so the impact of the two numerical protection items on the sorting results in normal calculation can be ignored.

[0097] The facial quality threshold and the visible iris area ratio threshold are selected using independent validation data. For each candidate threshold, the percentage of frames retained, the success rate of facial embedding extraction, the percentage of valid bits in the iris binary code, and the resampling ratio are statistically analyzed. The threshold combination that results in a facial embedding extraction success rate of at least 0.99%, a valid iris binary code ratio of at least 0.25, and the lowest resampling ratio is prioritized; if no combination meets these conditions, a quality threshold calibration failure status is output.

[0098] Candidate frames retain their continuous quality values ​​after passing the quality threshold, without binarizing them. Effective facial quality and effective iris texture quality are respectively weighted in the geometric mean of the cross-modal pose-consistent effective texture index and the quality-weighted average of the cross-modal fusion distance. The quality threshold only determines candidate eligibility and does not directly change the cross-modal fusion distance.

[0099] Processor 100 saves each candidate frame , , Visible iris area ratio, gradient amplitude ratio, and If any intermediate value is not a finite value or exceeds the defined range, the corresponding frame will be removed from the candidate set and the abnormal value will be recorded.

[0100] Example 6: This embodiment corresponds to Figure 1 The detection, localization and mask generation module 200, the cross-modal index calculation module 220 and the unique frame pair selection module 230 are used to explain the pose mismatch penalty, the cross-modal pose consistent effective texture index and the unique frame pair.

[0101] The cross-modal index calculation module 220 performs a combined enumeration of all candidate face frames and all candidate iris frames. For candidate face frames... With candidate iris frames A time-pose mismatch penalty is formed based on the exposure center time difference, pose difference, and projection difference. The facial head pose vector and the eye optical axis pose vector are both three-dimensional unit direction vectors in the 120 coordinate system of the facial visible light camera. The iris center is projected onto the facial field of view through the local plane homography matrix of the eye.

[0102] The cross-modal index calculation module 220 uses bounded normalization to calculate the pose mismatch penalty: ; in, and These are the exposure center moments for the candidate face frame and the candidate iris frame, respectively. and These are the corresponding exposure times. This is the facial head pose vector. This is the eye optical axis attitude vector. For the eye anchor points in the facial image, For the center of the iris, This represents the projection relationship from the iris field of view to the facial field of view. The pixel feature scale of the ROI for candidate face frames. , and Used to prevent the denominator from being zero.

[0103] The three components of the pose mismatch penalty are first mapped to the range of zero to one, and then their arithmetic mean is taken. As the exposure center time difference, pose difference, or projection difference increases, the corresponding component increases monotonically, and no single component will infinitely dominate the other two due to an excessively large numerical range. From this calculation, we can obtain... .

[0104] The cross-modal index calculation module 220 generates cross-modal pose-consistent effective texture indices for candidate frame pairs based on facial effective quality, iris effective texture quality, and pose mismatch penalty. ; in, The effective facial quality of candidate facial frames. The effective texture quality of the iris for candidate iris frames. This is the pose mismatch penalty for the corresponding candidate frame pair. From the input range, we can obtain... .

[0105] The unique frame pair selection module 230 sorts the candidate frame pairs according to a deterministic sorting order, and determines the first sorted pair as the unique frame pair: ; The sorting keys represent, in descending order: cross-modal pose-consistent effective texture index, exposure center time difference, effective iris texture quality, face frame index, and iris frame index. The frame indices are generated in ascending order of exposure center time when written to the acquisition buffer 160, thus ensuring a unique first-order sort for any candidate set.

[0106] If no candidate face frame or candidate iris frame exists, the unique frame selection module 230 outputs a capture failure status to the result image generation module 260. If the value is below the minimum available threshold, the unique frame pair selection module 230 outputs resampling trigger information and does not send the unique frame pair to the standardized feature extraction module 240.

[0107] Preferably, count values ​​1 to 10 of the unified hardware clock 110 are selected. Preferred selection The minimum usable threshold is preferably 0.3, but can be selected within the range of 0.2 to 0.5. The device uses verification data to enumerate candidate values ​​within this range, calculates the feature extraction failure rate and identity recognition error rate, selects the candidate value that simultaneously meets the device objective and has the largest value, and writes it into the configuration file.

[0108] If the camera calibration parameter version changes, the camera installation position changes, or any camera is repaired, the attitude transformation and projection relationship calibration will be re-executed. If the camera calibration parameter storage unit 150 has no available version, the homography matrix reprojection error does not meet the conditions, or the attitude vector normalization fails, the processor 100 will output a camera calibration abnormality status and will not calculate the attitude mismatch penalty.

[0109] The exposure center time difference component uses the fixed channel delay correction value obtained in Example 1. This component is calculated only when both exposure times are positive; if either exposure time is not greater than zero, the corresponding frame does not enter the candidate set. It only prevents division by zero when the product of two exposure times is abnormally close to zero, and does not replace the validity check of the exposure time.

[0110] The two inputs to the attitude difference term are both three-dimensional unit direction vectors. Before computation, the absolute difference between the vector's L2 norm and 1 is checked to ensure it does not exceed... If the conditions are not met, renormalize; if renormalization still fails, discard the corresponding frame. This sub-item compares the direction difference in the same coordinate system, but does not directly compare Euler angles, rotation matrices, and translation vectors.

[0111] The projection difference term places the eye anchor point and iris center projection positions in the facial image within the distortion-corrected facial field of view. ROI pixel feature scale. The eye-to-eye center distance is fixed and cannot be chosen between the face frame width and the eye-to-eye center distance. When the eye-to-eye center distance is less than 20 pixels, the corresponding facial frame is not included in the candidate set.

[0112] The three original mismatch ratios were respectively obtained through Mapped to the range of zero to one. This mapping is in When non-negative, it increases monotonically. As the value increases, it gradually approaches 1, preserving the ranking relationship while limiting extreme values ​​for individual items. All three mapping results use the same one-third coefficient, and no runtime variable weights are set to avoid a lack of basis for weighting.

[0113] The cross-modal index calculation module 220 enumerates candidate frame pairs using the face frame index as the outer loop and the iris frame index as the inner loop. Each frame pair records... , , , The record includes the exposure center time difference and two frame indices. Records missing any field or whose field values ​​are not finite are not added to the sorting list.

[0114] Deterministic ordering uses a fixed tolerance in numerical comparisons. Two cross-modal pose-consistent effective texture indices are considered equal if the absolute value of the difference does not exceed the tolerance. Then, the exposure center time difference is compared. If the exposure center time differences are equal, the effective texture quality of the iris is compared. If they are still equal, the face frame index and the iris frame index are compared in turn. The last two indices cannot be the same at the same time and represent different frame pairs, so the first and second indices are unique.

[0115] The minimum available threshold calibration uses validation data consistent with the device's camera, lighting, and model version. Validation data includes scenes with natural passage, slight head turns, blinking, eye glare, and single-sensor sequential exposures. Calibration is performed each time the camera position, lighting intensity, model version, or quality formula changes; thresholds from older configurations are not directly reused.

[0116] After a re-sampling is triggered, a new acquisition event number is used to avoid mixing the re-sampling frame with the first acquisition frame for enumeration. The second acquisition can use the same exposure parameters, or the device can select a different set of exposure parameters from a fixed configuration table based on the failure status of the first acquisition; each set of parameters in the configuration table is verified before deployment.

[0117] Example 7: This embodiment corresponds to Figure 1 The unique frame pair selection module 230 and the normalized feature extraction module 240 are used to illustrate the extraction of face embedding vectors, iris binary codes and effective bit masks based on unique frame pairs.

[0118] The standardized feature extraction module 240 receives the unique frame pairs output by the unique frame pair selection module 230. For the corrected facial image in a unique frame pair The scale is normalized according to the center of both eyes and the inner and outer corners of both eyes. Scale normalization adjusts the distance between the center of both eyes to 56 pixels, rotates the line connecting the two eyes to a horizontal position, and crops the image based on the face bounding box and the eye anchor points of the facial image. Standardized facial images.

[0119] The standardized feature extraction module 240 normalizes the pixel values ​​of the standardized facial image to the range of zero to one, and obtains the facial embedding vector through the facial embedding extraction model. The face embedding extraction model uses the following input: A pre-trained convolutional neural network that outputs a 512-dimensional vector from a three-channel image. The model output vector undergoes L2 normalization, and the model version is consistent with the model version of the 300 records in the registration and clearance template library.

[0120] For the corrected iris image in a unique frame pair The standardized feature extraction module 240 performs polar coordinate unfolding based on the pupil boundary and the outer iris boundary, unfolding the complete iris annulus into... Standardized iris strips. Bilinear interpolation is performed on the corrected iris image at each sampling location using linear interpolation coordinates between the pupil boundary point and the outer iris boundary point, and the validity of the sampling location is determined based on the visible iris mask, the specular occlusion mask, and the correction invalid pixels.

[0121] The standardized feature extraction module 240 performs directional filtering and phase quantization on the standardized iris strips to obtain the iris binary code. Directional filtering employs a pair of orthogonal-phase Gabor filters; at each sampling position, the symbols of the two filter responses are quantized into one code bit. The normalized feature extraction module 240 copies the valid state at each sampling position to the corresponding two code bits, generating a valid bit mask. The iris binary code and the effective bit mask length are both... .

[0122] The standardized feature extraction module 240 uses only the corrected facial image and the corrected iris image from a unique frame pair. If keypoints are missing, boundary interpolation fails, the proportion of invalid pixels in the standardized facial image exceeds 0.1, or the proportion of valid sampling positions in the standardized iris strip is less than 0.25 during the standardization process, the standardized feature extraction module 240 outputs a feature extraction failure status to the result image generation module 260 and triggers resampling.

[0123] The output of the standardized feature extraction module 240 includes facial embedding vectors. iris binary code Effective position mask Standardized facial images, standardized iris stripes, and effective facial quality corresponding to unique frame pairs. Effective iris texture quality Effective texture index consistent with cross-modal pose These outputs are fed into the fusion matching module 250; the normalized facial image, normalized iris strip, and effective position mask are also fed into the result image generation module 260.

[0124] The facial embedding extraction model is trained and frozen before device deployment, and its parameters are not updated during runtime. The registration template generation end and the border inspection channel acquisition end use the same input size, pixel normalization, output dimension, and model version. Model files are verified using hash values; if the hash value does not match one of the 300 records in the registration and clearance template library, an abnormal model version status is output.

[0125] The preferred center frequency of the Gabor filter is 0.25 Hz, and the preferred bandwidth is one octave. The standardized iris strips are cyclical along the angular direction but not along the radial direction. Filter parameters, standardized iris strip dimensions, and code mapping rules are written into the template version information of the registration and clearance template library 300. Iris distance calculation is not performed if the template version differs from that of the acquisition terminal.

[0126] Facial scale normalization uses a similarity transform. The scaling ratio and rotation angle are determined based on the binocular centers, and the translation amount is determined based on the eye anchor points of the facial image. The corrected facial image is then mapped to the normalized facial image. The similarity transform parameters are calculated from the binocular centers and eye anchor points of the facial image within the same frame, without using keypoints from other frames. Pixels whose transformed sampled coordinates exceed the limits are marked as invalid.

[0127] The input channel order, pixel mean, and pixel scaling factor of the face embedding extraction model are recorded in the model version file. The normalized face image is then processed according to this file to complete the channel arrangement and numerical transformation. The model output is not 512-dimensional, contains non-finite values, or has a L2 norm less than [a certain value]. When this happens, the output shows a feature extraction failure status. The length of the face embedding vector after L2 normalization is fixed at 512.

[0128] The template facial embedding vectors in the registration clearance template library 300 are generated using the same model version and subjected to the same L2 normalization. During the registration phase, multiple frames of facial images can be acquired, and the facial embedding vectors are extracted from each image. The arithmetic mean of these vectors is then calculated, and L2 normalization is performed again to form the template facial embedding vector. Each frame used for averaging must meet the same facial quality threshold as the one acquired by the border inspection system.

[0129] The iris polar coordinate unfolding determines the pupil boundary point and the outer iris boundary point for each angular sampling column, and then samples 64 equally spaced radial positions between the two points. The 512 angular columns cover a complete circle, with column 0 adjacent to column 511. When the pupil boundary or outer iris boundary has no effective intersection point in a certain angular direction, all sampling positions in that angular direction are recorded as 0 in the effective position mask.

[0130] The real and imaginary responses of the Gabor filter are compared to zero. A response greater than or equal to zero corresponds to a 1, and a response less than zero corresponds to a 0. A sampling location generates two code bits, and the valid bit mask copies the valid state of that sampling location twice. When the filter window covers more than half of the invalid sampling locations, both code bits generated at that location are recorded as 0 in the valid bit mask.

[0131] The iris binary code angular cyclic shift is performed column-by-column, shifting all radial code bits and the effective bit mask corresponding to each column. The fusion matching module 250 searches within a range of -8 to +8 columns, covering angular shifts caused by slight eye rotation. The search range, strip size, and code bit order are fixed within the template version.

[0132] The standardized facial image, standardized iris stripe, facial embedding vector, iris binary code, and effective bit mask are all associated with two frame indices of a unique frame pair. When the fusion matching module 250 receives data, it checks the frame indices; if any data source is inconsistent with the unique frame pair, it outputs an abnormal status associated with the feature data.

[0133] Example 8: This embodiment corresponds to Figure 1 The standardized feature extraction module 240, fusion matching module 250, result image generation module 260, registration and clearance template library 300, and border inspection rapid clearance biometric recognition image 400 are used to illustrate the generation of cross-modal fusion distance and identity recognition results.

[0134] The fusion matching module 250 receives the face embedding vector output by the standardized feature extraction module 240. iris binary code Effective position mask Effective facial quality Effective iris texture quality Effective texture index consistent with cross-modal pose The registration and clearance template library 300 provides the fusion matching module 250 with template facial embedding vectors, template iris binary codes, and template valid bit masks.

[0135] For templates in the registration and customs clearance template library 300 The fusion matching module 250 compares the facial embedding vector with the template facial embedding vector to obtain the facial distance normalized to the range of zero to one. ; in, For template facial embedding vectors, Used to process zero vectors. The normalized feature extraction module 240 has performed L2 normalization on the face embedding vectors; when the current vector or template vector is detected as a zero vector, the corresponding template is marked as unavailable.

[0136] The fusion matching module 250 performs a cyclic shift comparison between the iris binary code and the valid bit mask and the template iris binary code and the template valid bit mask to obtain the iris distance normalized to the range of zero to one. ; in, This is a template iris binary code. This is the effective bit mask for the template. Iris code number, The number of columns shifted cyclically in the angular direction. This is used to handle cases where the number of common valid code bits is zero. Both the iris binary code and the valid bit mask are binary arrays of the same length.

[0137] The fusion matching module 250 obtains the cross-modal fusion distance of each available template based on facial distance, iris distance, effective facial quality corresponding to a unique frame pair, effective iris texture quality, and cross-modal pose-consistent effective texture index. ; in, To prevent the denominator from being zero, both facial distance and iris distance are limited to a range of zero to one. , and All are limited to the range of zero to one, factor It falls within the range of one to two. Only when... , Furthermore, the cross-modal fusion distance is calculated only when the number of common valid code points reaches one-quarter of the total number of iris code points.

[0138] The fusion matching module 250 selects the template index with the smallest cross-modal fusion distance: ; The resulting image generation module 260 will... Compare with the identity determination threshold. When When the identity determination threshold is not exceeded, the identity recognition result is the same as the template. Same; when When the identity determination threshold is exceeded, the identity recognition result is that no matching of the same template is found.

[0139] The result image generation module 260 generates a border inspection rapid clearance biometric recognition image 400. The border inspection rapid clearance biometric recognition image 400 includes a standardized facial image, a standardized iris strip, an effective mask outline, and identity recognition results, and is associated with the acquisition event number and the result generation time. This image is used for system recording, channel device status display, or manual verification.

[0140] When the registration template library 300 is empty, the template data is corrupted, the template model version is inconsistent, or the left and right eye identifiers of the template are inconsistent with the current iris data, the fusion matching module 250 will mark the corresponding template as unavailable. When the number of common valid code points between the template's valid bit mask and the current valid bit mask is less than one-quarter of the total number of iris code points, the corresponding template will be marked as unavailable, and backtracking calculation using only facial distance will not be used.

[0141] when When the data falls below the minimum usable threshold, the processor 100 controls the face visible light camera 120 and the iris near-infrared camera 130 to re-acquire short facial and iris sequences within a short window, or controls the single-sensor sequential exposure device 140 to re-acquire face and iris frames. The supplementary data is written to a new acquisition event, and the same processing flow is executed. If two consecutive acquisitions fail to meet the minimum usable threshold, an insufficient acquisition quality status is output.

[0142] Preferred selection , The preferred value is 1. Preferred selection The threshold for the number of common valid code points is preferably one-quarter of the total number of iris code points, and can be selected and fixed within the range of one-fifth to one-third based on the stability of iris distance in the verification data.

[0143] Registration and clearance templates are generated during the registration phase. The registration process uses the same facial embedding extraction model, iris strip size, Gabor filter parameters, and code point mapping rules as the border inspection channel. Each template records the template number, facial embedding vector, iris binary code, template valid bit mask, left and right eye identifiers, model version, template version, and integrity check value. Templates that fail the integrity check are not included in the matching process.

[0144] The identity determination threshold is calibrated using independent verification data before device deployment. The verification data includes pairings of the same person and pairings of different people, and the cross-modal fusion distance is calculated completely according to this implementation method; within the range of zero to two... The step size is used to enumerate candidate thresholds, and the selected candidate has a false acceptance rate not exceeding [a certain value]. The candidate threshold with the lowest false rejection rate is selected. If no candidate threshold meets the condition, a threshold calibration failure status is output, and identity recognition is not enabled.

[0145] When the candidate template set is empty or all templates are marked as unavailable, the fusion matching module 250 outputs a template unavailable status. When multiple templates have the same minimum cross-modal fusion distance, they are determined in ascending order of template number. This ensures that each calculation yields a unique output.

[0146] Both the template face embedding vector and the current face embedding vector are normalized to L2, therefore the cosine similarity is in the range of -1 to 1. The face distance is multiplied by half outside the cosine distance, making the face distance range of zero to 1. Zero vectors are not included in the formula calculation. It is reserved only as a value protection item.

[0147] The iris distance calculation first checks if the lengths of the current iris binary code, the template iris binary code, the current valid bit mask, and the template valid bit mask are consistent. If the lengths are inconsistent, the template is marked as unusable. For each cyclic shift... Calculate the number of common valid code points and the number of different bits on the common valid code points respectively; if the number of common valid code points is lower than the threshold, the shift result is unusable. If at least one shift result is usable, take the minimum iris distance.

[0148] When the value is 1, the impact on iris distance under the condition of a large number of common valid code points does not exceed the proportion corresponding to one code point. Results with zero or less than the threshold of common valid code points have been excluded and are not utilized. Invalid results are converted into usable results. This process avoids obtaining zero iris distance when there are no valid bits.

[0149] Cross-modal fusion distance is calculated only for templates that simultaneously possess both usable facial distance and usable iris distance. If the effective facial quality or effective iris texture quality is not greater than zero, the cross-modal pose-consistent effective texture index is below the minimum usable threshold, the template data version is inconsistent, or the number of common effective code points is insufficient, the corresponding template will not generate cross-modal fusion distance. Processor 100 does not use an alternative formula that retains only one modality.

[0150] The sum of the two effective qualities should be kept sufficiently small. The sum of the two effective qualities should not be less than the sum of the two quality thresholds at deployment time, therefore, in normal calculations... It does not determine the quality weight. The smaller the effective texture exponent for cross-modal pose consistency, the closer the modulation factor is to 2; the larger the exponent, the closer the modulation factor is to 1, and the modulation factor is always positive.

[0151] When calibrating the identity determination threshold, both pairings of the same person and pairings of different people are generated using independent acquisition events, without directly comparing the registered image with itself. For each candidate threshold, the number of false acceptances, false rejections, the total number of pairings of different people, and the total number of pairings of the same person are counted. The false acceptance rate and false rejection rate are calculated proportionally to their respective numbers, and the selection results, validation data version, model version, and camera calibration parameter version are all written into the configuration file.

[0152] Integrity checks are performed when the configuration file is loaded. If any of the following thresholds are missing: minimum available threshold, facial quality threshold, visible iris area ratio threshold, number of common valid code points threshold, or identity determination threshold, processor 100 outputs a configuration error status. These thresholds remain unchanged during operation and are not dynamically adjusted based on individual passengers, individual templates, or current distance.

[0153] The border inspection fast-clearance biometric recognition image 400 uses a fixed four-region layout, displaying a standardized facial image, a standardized iris strip, a valid position mask outline, and the identity recognition result. If a standardized image cannot be generated, a corresponding acquisition failure status is displayed, and images from other passengers or other acquisition events are not used to fill the gap. The image file name or database record key includes the acquisition event number to correspond with the calculation record.

[0154] The results record should include at least the acquisition event number, a unique frame pair index, and , , , , , , The records include the identity determination threshold, template number, and identity identification result. The values ​​recorded are only used for technical traceability and equipment verification and do not alter the already completed identity identification results.

[0155] Template data corruption, model version abnormality, camera calibration abnormality, clock synchronization abnormality, acquisition failure, processing timeout, feature extraction failure, threshold calibration failure, and insufficient acquisition quality are all output as independent technical statuses. Only when a valid cross-modal fusion distance is generated and threshold comparison is completed will an identity recognition result be output, indicating whether the same template or not matched. Other statuses are not rewritten as identity recognition results.

[0156] An end-to-end configuration is loaded upon device startup, including a unified hardware clock counting frequency, two types of camera calibration parameter versions, line exposure center delay mapping, facial keypoint detection model version, iris segmentation model version, face embedding extraction model version, iris encoding parameters, various quality thresholds, minimum usable threshold, common valid code point number threshold, and identity determination threshold. The device enters the data acquisition state after all items pass integrity verification; if any item is missing, the device remains in an abnormal configuration state.

[0157] After a passenger enters the optical acquisition area, acquisition buffer 160 creates an acquisition event number and receives two types of short sequences. After the acquisition window closes, frames are first filtered by time information and camera calibration parameter version, and then detection and localization, motion degradation processing, and quality filtering are performed. Candidate frame pairs are enumerated only when both candidate sets are not empty. Standardized feature extraction is performed only when a unique frame pair is generated and the effective texture index with consistent pose across modalities reaches the minimum usable threshold.

[0158] After standardized feature extraction, the fusion matching module 250 first forms a set of usable templates based on template version, model version, left and right eye identifiers, and integrity check values. For each usable template, the facial distance and the iris distance corresponding to each cyclic shift are calculated; only shift results that meet the condition of having a sufficient number of common valid code points participate in the minimum value selection. Cross-modal fusion distance is calculated only when both facial distance and iris distance are available.

[0159] After the cross-modal fusion distance of all available templates is calculated, the fusion matching module 250 sorts them in ascending order of cross-modal fusion distance and template number. The cross-modal fusion distance of the first template in the sort is compared with the identity determination threshold. The result image generation module 260 simultaneously receives the sorting results, the standardized image of the unique frame pair, and the effective bit mask to form the output record of the same acquisition event and the biometric recognition image 400 for rapid border inspection clearance.

[0160] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A biometric scanning and recognition system for rapid border inspection clearance, comprising acquiring short facial and iris sequences within a single acquisition event, characterized in that, Configured for execution: The exposure center time is determined for each frame in the facial short sequence and the iris short sequence based on a unified hardware clock. A candidate face frame set and a candidate iris frame set are formed from the short facial sequence and the short iris sequence, respectively. The cross-modal pose-consistent effective texture index of candidate frame pairs is calculated based on the effective facial quality of candidate face frames, the effective iris texture quality of candidate iris frames, and the pose mismatch penalty between candidate face frames and candidate iris frames. A unique frame pair is determined based on the cross-modal pose-consistent effective texture index; Based on the unique frame pair, standardized feature extraction and fusion matching are performed, and the cross-modal fusion distance is modulated according to the cross-modal pose-consistent effective texture index corresponding to the unique frame pair. The identity recognition result is output according to the cross-modal fusion distance.

2. The biometric scanning and identification system for rapid border inspection clearance according to claim 1, characterized in that, The acquisition of short facial and iris sequences within a single acquisition event includes: After passengers enter the optical acquisition area of ​​the border inspection channel, the facial visible light camera and the iris near-infrared camera are activated simultaneously to acquire short-sequence data, or facial frames and iris frames are acquired by a single sensor in the order of exposure and illumination switching. The exposure time, frame interval, and line exposure center delay mapping are recorded for each frame, and each frame is associated with camera calibration parameters, which include camera intrinsic parameters and camera extrinsic parameters.

3. The biometric scanning and identification system for rapid border inspection clearance according to claim 2, characterized in that, Forming candidate facial frame sets and candidate iris frame sets from the short facial sequence and the short iris sequence respectively includes: Extract the face bounding box, inner and outer corners of the eyes, center of both eyes, eye anchor points, eye center distance, occlusion area, and facial head pose vector from the face frame; Extract the pupil boundary, outer iris boundary, eyelid boundary, iris center, area of ​​complete iris ring, mask of complete iris ring, area of ​​visible iris, mask of visible iris, specular occlusion mask, and eye optical axis pose vector from the iris frame; The facial head pose vector and the eye optical axis pose vector are converted to a comparable vector space. The projection relationship from the iris field of view to the facial field of view is established based on the camera calibration parameters. The complete face frames and iris frames with the detection and localization results are added to the candidate face frame set and the candidate iris frame set, respectively.

4. The biometric scanning and identification system for rapid border inspection clearance according to claim 3, characterized in that, Motion degradation processing of the candidate face frame set and the candidate iris frame set includes: Optical flow within a biometric region of interest (ROI) is calculated between adjacent frames in the same short sequence. The average pixel velocity of the ROI is obtained based on the difference in exposure center time between adjacent frames and the median of the horizontal and vertical components of the optical flow. Motion degradation is obtained based on exposure time, the average pixel velocity of the ROI, and the pixel feature scale of the ROI. The original image is then resampled based on the row exposure center delay mapping and the average pixel velocity of the ROI to obtain the corrected face image and the corrected iris image. Frames whose effective optical flow ratio does not meet the motion degradation processing conditions are removed from the corresponding candidate set.

5. The biometric scanning and identification system for rapid border inspection clearance according to claim 4, characterized in that, The effective facial quality is calculated by a normalized facial quality scalar formed by the integrity of the face frame, key point confidence, exposure quality, occlusion ratio, and pose availability of the candidate facial frame, and the motion degradation of the candidate facial frame; the effective iris texture quality is calculated by the ratio of the visible iris area of ​​the candidate iris frame to the area of ​​the complete iris ring, the ratio of the gradient magnitude of the corrected iris image within the visible iris mask to the gradient magnitude within the complete iris ring mask, and the motion degradation of the candidate iris frame.

6. The biometric scanning and identification system for rapid border inspection clearance according to claim 5, characterized in that, The pose mismatch penalty is formed based on the exposure center time difference between the candidate face frame and the candidate iris frame, the pose difference between the face head pose vector and the eye optical axis pose vector in the comparable vector space, and the projection difference between the iris center projected onto the face field of view through the projection relationship and the eye anchor point of the face image. The exposure center time difference, the pose difference and the projection difference are respectively bounded normalized, and the arithmetic mean of the three normalization results is taken as the pose mismatch penalty.

7. The biometric scanning and identification system for rapid border inspection clearance according to claim 6, characterized in that, The cross-modal pose-consistent effective texture index is formed based on the effective facial quality, the effective iris texture quality, and the pose mismatch penalty. All candidate frame pairs are sorted in the following order: cross-modal pose-consistent effective texture index from largest to smallest, exposure center time difference from smallest to largest, effective iris texture quality from largest to smallest, facial frame index from smallest to largest, and iris frame index from smallest to largest. The candidate frame pair with the highest ranking is determined as the unique frame pair, and the effective facial quality, effective iris texture quality, and cross-modal pose-consistent effective texture index corresponding to the unique frame pair are output.

8. The biometric scanning and identification system for rapid border inspection clearance according to claim 7, characterized in that, The standardized feature extraction based on the unique frame pair includes: Using only the corrected facial image in the unique frame pair, the scale is normalized according to the center of both eyes and the inner and outer corners of both eyes to obtain the facial embedding vector; Using only the corrected iris image in the unique frame pair, polar coordinates are expanded based on the pupil boundary and the outer iris boundary, and the visible iris mask and the specular occlusion mask are combined to generate the iris binary code and the effective bit mask.

9. The biometric scanning and identification system for rapid border inspection clearance according to claim 8, characterized in that, The fusion matching includes: The facial embedding vector is compared with the template facial embedding vector in the registration and clearance template library to obtain the facial distance normalized to the range of zero to one. The iris binary code and the effective bit mask are cyclically shifted and compared with the template iris binary code and the template effective bit mask to obtain the iris distance normalized to the range of zero to one. When the number of common effective code points reaches one-quarter of the total number of iris code points and the effective facial quality and effective iris texture quality corresponding to the unique frame pair are both greater than zero, the cross-modal fusion distance of each template is obtained based on the facial distance, the iris distance, the effective facial quality, the effective iris texture quality and the cross-modal pose-consistent effective texture index. Select the template with the smallest cross-modal fusion distance, compare the cross-modal fusion distance corresponding to the template with the identity determination threshold, and output the identity recognition result and the biometric recognition image for rapid border inspection clearance.

Images