Multi-modal biological characteristic acquisition method and all-in-one machine

Abstract

The invention discloses a multi-modal biological characteristic acquisition method and an all-in-one machine, and the method comprises the steps: guiding a user to carry out face pre-positioning through a face positioning frame displayed by a main display screen and face positioning prompt information displayed by an auxiliary display screen; controlling a main display screen to display an iris positioning ring and a virtual glasses frame, displaying iris positioning prompt information through an auxiliary display screen, and guiding a user to coincide the virtual glasses frame with the iris positioning ring by moving the head; the method comprises the following steps: respectively monitoring a face image and an iris image of a user through a telescope type iris acquisition module and a wide-angle face acquisition camera, and evaluating quality scores of the images; and capturing the face image and the iris image of the user at the same moment under the condition that the quality scores of the initially acquired face image and the iris image are both greater than the corresponding threshold values. According to the method, seamless integration of a user multi-modal biological characteristic acquisition process, high self-service of an acquisition process and real-time controllability of acquisition quality can be realized.

Description

Technical Field

[0001] This application relates to the field of biometric recognition technology, and in particular to a multimodal biometric acquisition method and an all-in-one machine. Background Technology

[0002] Currently, in fields such as security and identity authentication, the joint collection of multimodal biometrics (such as iris and face) has become an important means to improve recognition accuracy and security. By integrating multiple biometrics, the shortcomings of single biometric recognition in terms of stability and security can be effectively compensated, meeting the application requirements of high-security scenarios.

[0003] However, current multimodal biometric acquisition schemes typically collect one biometric feature at a time during each data acquisition process, dividing multimodal biometrics into multiple acquisition steps. This approach fragments the acquisition process, resulting in low efficiency, reliance on manual guidance, and poor data quality. Summary of the Invention

[0004] The purpose of this application is to at least partially solve one of the aforementioned technical problems.

[0005] Therefore, the first objective of this application is to propose a multimodal biometric acquisition method that can achieve seamless integration of the user's multimodal biometric acquisition process, a high degree of self-service in the acquisition process, and real-time controllability of the acquisition quality, thereby solving problems such as fragmented multimodal biometric acquisition processes, reliance on manual labor, and uncontrollable quality.

[0006] The second objective of this application is to propose a multimodal biometric acquisition device.

[0007] The third objective of this application is to provide a computer-readable storage medium.

[0008] To achieve the above objectives, the first aspect of this application proposes a multimodal biometric acquisition method, comprising the following steps: The system guides users to pre-position their faces by displaying a face positioning frame on the main screen and face positioning prompts on the secondary screen. After detecting that the user's face image is within the range of the face positioning frame, the main display screen is controlled to display the iris positioning circle and the virtual glasses frame that is linked to the user's facial posture, and the iris positioning prompt information is displayed through the secondary display screen to guide the user to move their head to align the virtual glasses frame with the iris positioning circle so that the user's eyes are located at the iris acquisition target position. The system uses a telescopic iris capture module and a wide-angle face capture camera to monitor the user's initial face image and initial iris image, and evaluates the quality scores of the initial face image and initial iris image. If the quality scores of the initial face image and the initial iris image are both greater than the corresponding thresholds, capture the user's face image and iris image at the same time.

[0009] In addition, the multimodal biometric acquisition method of this application embodiment also has the following additional technical features: Optionally, in some embodiments, the face positioning frame displayed on the main display screen and the face positioning prompt information displayed on the secondary display screen guide the user to perform face pre-positioning, including displaying prompt information on the secondary display screen guiding the user to face the main display screen; monitoring the user's face information in real time through the wide-angle face acquisition camera and detecting the real-time distance between the user and the main display screen through a distance sensor; and guiding the user to adjust their position through position adjustment prompt information displayed on the secondary display screen when the real-time distance is greater than a distance threshold, until the user's face image is within the range of the face positioning frame.

[0010] Optionally, in some embodiments, controlling the main display screen to display the iris positioning circle and the virtual glasses frame linked to the user's facial posture includes: displaying the iris positioning circle at the center of the main display screen, and displaying the virtual glasses frame corresponding to the user's current facial posture on the main display screen based on facial key point detection technology; the step of displaying iris positioning prompt information through the secondary display screen includes: displaying prompt information on the secondary display screen guiding the user to move their head to align their eyes with the iris positioning circle, wherein, at the iris acquisition target position, the physical light axis of the telescopic iris acquisition module corresponds to the position of the user's eyes.

[0011] Optionally, in some embodiments, evaluating the quality scores of the initial face image and the initial iris image includes: analyzing the sharpness of the initial iris image, determining whether there are obstructions in the initial iris image, and calculating the pupil center offset; calculating the quality score of the initial iris image based on the sharpness, the obstruction status of the obstruction, and the pupil center offset; and calculating the quality score of the initial face image based on the angle information, illumination information, and facial feature status information of the face in the initial face image.

[0012] Optionally, in some embodiments, after evaluating the quality scores of the initial face image and the initial iris image, the method further includes: if the quality score of the initial face image or the initial iris image fails the evaluation, re-displaying guidance prompts for the image that failed the evaluation on the sub-display; and cyclically performing quality evaluation and adaptive adjustment based on the updated face image and iris image.

[0013] Optionally, in some embodiments, after capturing the user's face image and iris image at the same time, the method further includes: packaging the captured face image and iris image with the user's identity information, fingerprint information, and high-definition photo data collected by an external device; and printing the packaged data using a thermal printer to output a business credential.

[0014] To achieve the above objectives, a second aspect of this application proposes a multimodal biometric data acquisition device, comprising: a housing and a main display screen, a secondary display screen, a telescopic iris acquisition module, a wide-angle face acquisition camera, and a control unit disposed within the housing; wherein, The main display screen is used to display animated images required for face localization and iris localization; The secondary display screen is used to display prompts that guide the user to perform face and iris localization. The telescope-type iris acquisition module is used to acquire the user's facial image; The wide-angle face capture camera is used to capture the user's iris image; The control unit establishes communication connections with each module in the housing, and the control unit is used to coordinate the control of each module to realize the multimodal biometric acquisition method as described in the first aspect above.

[0015] Optionally, the all-in-one machine also includes a distance sensor for detecting the real-time distance between the user and the main display screen.

[0016] Optionally, the all-in-one machine also includes: an ID card reader for collecting the user's identity information; a fingerprint scanner for collecting the user's fingerprint information; a high-resolution document scanner for collecting high-definition photos of items carried by the user; and a thermal printer for printing various data collected by the multimodal biometric data collection all-in-one machine and outputting business vouchers.

[0017] To achieve the above objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal biometric acquisition method as described in any one of the first aspects of the embodiments above.

[0018] The technical solutions provided by the embodiments of this application have at least the following beneficial effects: This application significantly improves data acquisition efficiency and user experience. Through graphical collaborative guidance and visual linkage design on the main and secondary displays, it integrates the previously fragmented multi-step processes of face and iris recognition into a smooth, intelligent process. Users no longer need to repeatedly move their bodies to adjust distance, shortening the average acquisition time. The entire process requires no human intervention; users can complete the operation completely autonomously, reducing labor costs and avoiding operational discrepancies caused by human guidance, achieving an efficient and user-friendly self-service acquisition experience. Furthermore, this application significantly enhances data acquisition quality and reliability. By performing real-time dual-modal image quality assessment during acquisition and providing precise adaptive feedback based on the assessment results, it guides users to adjust their posture in a timely manner, preventing the entry of unqualified data into the database from the source. High-quality acquired data ensures the accuracy and stability of subsequent identification and authentication processes, meeting the application requirements for applications with strict standards for biometric data. Moreover, this application improves the intelligence and integration level of the device, deeply combining hardware features with software algorithms, fully leveraging the integrated advantages of the all-in-one machine. Meanwhile, the device supports the expansion of various optional peripherals and additional bio-information acquisition modules, and can be customized according to the needs of different application scenarios, thus broadening its application prospects in different fields.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the structure of a multimodal biometric data acquisition device proposed in an embodiment of this application; Figure 2 This is a flowchart of a multimodal biometric acquisition method proposed in an embodiment of this application; Figure 3 This is a schematic diagram illustrating the implementation principle of a multimodal biometric feature acquisition method proposed in this application embodiment. Detailed Implementation

[0021] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0022] It should be noted that the multimodal biometric data acquisition methods in the relevant embodiments have the following drawbacks, which seriously affect acquisition efficiency, user experience, and data quality: First, the acquisition process is fragmented, resulting in a poor user experience. Iris acquisition requires the user to precisely place their eyes close to a dedicated telescope-like acquisition window (the optimal distance is about 15-20cm), while face acquisition requires the user to maintain a greater distance from the device (usually 50-100cm) and face the camera directly. This difference in distance requirements forces users to repeatedly and blindly move their bodies to find two different acquisition points in actual operation. The entire acquisition process is disjointed, causing operational confusion for users and significantly reducing acquisition efficiency.

[0023] Second, it relies heavily on manual guidance and has a low degree of automation. Due to the complexity of the data collection process and the variability of operational requirements, the entire process heavily depends on verbal guidance and correction from on-site staff. This not only increases labor costs but may also lead to inconsistent data collection due to human factors such as varying operating standards and differing guidance expressions, affecting the universality of subsequent recognition algorithms.

[0024] Third, the lack of real-time quality feedback results in unreliable data quality. The acquisition systems in the relevant embodiments typically only assess image quality after the user has completed the acquisition process. If the acquired image is substandard (e.g., blurred iris, occlusion, incorrect facial angle, insufficient lighting, etc.), the user must cooperate to perform a second acquisition. This not only wastes the time of users and staff but may also lead to low-quality data entering the database at the source, severely impacting the performance of subsequent recognition algorithms and potentially causing recognition failures or misidentifications.

[0025] Therefore, this application proposes a multimodal biometric data acquisition device and its application method. This application can intelligently guide users to collect data and complete high-quality acquisition of multimodal biometrics through an integrated acquisition method.

[0026] The multimodal biometric acquisition method and integrated machine of this application are described below with reference to the accompanying drawings. To implement the multimodal biometric acquisition method of this application, a multimodal biometric acquisition integrated machine is proposed. To facilitate understanding of the implementation process and principle of the multimodal biometric acquisition method of the first aspect embodiment of this application, the multimodal biometric acquisition integrated machine proposed in the second aspect embodiment of this application will be described first.

[0027] Figure 1 This is a schematic diagram of the structure of a multimodal biometric data acquisition device proposed in an embodiment of this application, as shown below. Figure 1As shown, the all-in-one device includes: a housing and a main display screen 10, a secondary display screen 20, a telescope-type iris acquisition module 30, a wide-angle face acquisition camera 40, and a control unit 50 integrated in the housing.

[0028] The main display screen 10 is used to display animated images required for face and iris localization.

[0029] The secondary display screen 20 is used to display prompts that guide the user through face and iris localization.

[0030] The telescope-type iris acquisition module 30 is used to acquire the user's facial image.

[0031] The wide-angle face capture camera 40 is used to capture the user's iris image.

[0032] The control unit 50 establishes communication connections with each module in the housing, and the control unit 50 is used to coordinate the control of each module to realize the multimodal biometric acquisition method proposed in the first aspect of this application.

[0033] Among them, such as Figure 1 As shown, the multimodal biometric acquisition method of this application corresponds to the software module in the all-in-one machine, and the relevant algorithms in the software module are implemented based on the hardware platform in the all-in-one machine.

[0034] The control unit 50 is electrically connected to the main display screen 10, the secondary display screen 20, the telescopic iris recognition module 30, the wide-angle face recognition camera 40, and other modules in the all-in-one system. It controls the collaborative operation of these modules to achieve various functions such as data acquisition, processing, and transmission. The wide-angle face recognition camera 40 can be positioned above the main display screen 10, enabling it to better capture images of the user's frontal face.

[0035] In one embodiment of this application, the multimodal biometric acquisition device further includes a distance sensor 60, which is used to detect the real-time distance between the user and the main display screen 10.

[0036] In this embodiment, the control unit 50 can use the distance sensor 60 to perform face positioning related control based on the detected distance data.

[0037] In one embodiment of this application, the all-in-one machine further includes: an ID card reader for collecting the user's identity information; a fingerprint collector for collecting the user's fingerprint information; a high-resolution document scanner for collecting high-definition photo data of items carried by the user; and a thermal printer for printing various data collected by the multimodal biometric data collection all-in-one machine and outputting business vouchers.

[0038] Specifically, such as Figure 1 As shown, the all-in-one machine in this embodiment also includes optional peripherals. Each optional peripheral establishes a communication connection with the control unit 50. Each optional peripheral includes, but is not limited to, at least one of: ID card reader, fingerprint collector, high-speed scanner, and thermal printer.

[0039] Among these, ID card readers can be used to obtain users' identity text information, enabling the binding of biometric features with identity information. Fingerprint collectors can expand the collection of users' fingerprint biometric features, further improving the accuracy and security of identification. High-speed document scanners can be used to collect users' ID photos or other relevant document images, quickly capturing paper documents, ID cards, and physical objects carried by users through a high-definition camera and converting them into digital images. Thermal printers can instantly output business vouchers, providing users with written proof of completed data collection and improving the completeness of business processing.

[0040] The following is a detailed description of the multimodal biometric acquisition method proposed in this application, based on the aforementioned multimodal biometric acquisition all-in-one machine. Specifically, it involves controlling the relevant modules in the all-in-one machine of the above embodiments to implement the method of this embodiment. The various devices involved in this method are as described in the above embodiments and will not be repeated here. The execution entity of this method can be the aforementioned control unit 50.

[0041] Figure 2 This is a flowchart of a multimodal biometric acquisition method proposed in an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps: Step S101: Guide the user to perform face pre-positioning by using the face positioning frame displayed on the main display screen and the face positioning prompt information displayed on the secondary display screen.

[0042] Specifically, this step involves initial guidance and face pre-positioning. The user is guided to move continuously by the face positioning frame displayed on the main screen and the face positioning prompts displayed on the secondary screen. The captured user face image is then displayed on the main screen and adjusted to fit within the face positioning frame displayed on the main screen, thereby achieving face pre-positioning.

[0043] In one embodiment of this application, a user is guided to pre-position their face using a face positioning frame displayed on the main display screen and face positioning prompts displayed on the secondary display screen. This includes: displaying prompts on the secondary display screen guiding the user to face the main display screen; monitoring the user's face information in real time using a wide-angle face capture camera and detecting the real-time distance between the user and the main display screen using a distance sensor; and guiding the user to adjust their position using position adjustment prompts displayed on the secondary display screen when the real-time distance is greater than a distance threshold, until the user's face image is within the range of the face positioning frame.

[0044] For example, after the all-in-one device is powered on, the main display shows a clear animation of a face positioning frame, attracting the user's attention through visual guidance. Simultaneously, the secondary display shows text prompts, such as "Please face the main screen," clearly informing the user of the initial operating requirements. Then, the wide-angle face capture camera and distance sensor activate simultaneously. The wide-angle face capture camera detects the user's facial position in real time, while the distance sensor detects the distance between the user and the device. When the user is too far away, the secondary display promptly prompts "Please move closer to the screen," guiding the user to adjust their position. When the user enters the effective distance range (e.g., 30 to 100 cm from the main display) and the user's facial image is within the face positioning frame, the system determines that the user is in a suitable initial capture position and enters a ready state, providing a foundation for subsequent iris capture alignment.

[0045] Step S102: After detecting that the user's face image is within the range of the face positioning box, control the main display screen to display the iris positioning circle and the virtual glasses frame that is linked to the user's facial posture, and display the iris positioning prompt information through the secondary display screen to guide the user to move their head to align the virtual glasses frame with the iris positioning circle so that the user's eyes are located at the iris acquisition target position.

[0046] Specifically, this step involves coordinated visual guidance and iris capture alignment. When the user's face is determined to be within the capture range based on the image captured by the wide-angle face capture camera, the main display screen automatically switches to display the iris positioning circle and a virtual glasses frame corresponding to the user's real-time facial posture. Guided by the iris positioning prompts displayed on the secondary display screen, the user moves their head to align the virtual glasses frame with the iris positioning circle. In this state, the user's eyes are in the optimal position for iris capture (i.e., the target capture position).

[0047] In one embodiment of this application, controlling the main display screen to display an iris positioning circle and a virtual glasses frame linked to the user's facial posture includes: displaying the iris positioning circle at the center of the main display screen, and displaying a virtual glasses frame corresponding to the user's current facial posture on the main display screen based on facial key point detection technology; displaying iris positioning prompt information through a secondary display screen includes: displaying prompt information on the secondary display screen guiding the user to move their head to align their eyes with the iris positioning circle, wherein, at the iris acquisition target position, the physical light axis of the telescopic iris acquisition module corresponds to the position of the user's eyes.

[0048] For example, after the main display screen automatically switches interfaces, a clear iris positioning circle is displayed in the center of the screen as the target reference for iris acquisition. Simultaneously, using facial landmark detection technology, a virtual glasses frame icon that synchronizes with the user's facial posture is overlaid on the main display screen in real time. At this point, the prompt on the secondary display screen updates to: "Please move your head and align your eyes with the circle on the screen."

[0049] Therefore, this design precisely aligns the fixed physical optical axis of the telescope-style iris scanning module with the dynamic virtual positioning diagram on the main display screen. Users do not need to memorize complex distance requirements and operating procedures; they can simply align the virtual eyeglass frames representing their eyes with the iris positioning circle through intuitive observation. This naturally and accurately positions their eyes on the optimal iris scanning position along the physical optical axis of the iris scanning module. Thus, this embodiment achieves seamless integration of face pre-positioning and iris scanning alignment, resolving the operational confusion caused by distance switching in traditional scanning processes.

[0050] Step S103: Using a telescope-type iris acquisition module and a wide-angle face acquisition camera, monitor the user's initial face image and initial iris image respectively, and evaluate the quality scores of the initial face image and initial iris image.

[0051] Specifically, this step involves real-time quality assessment and adaptive feedback of the acquired images. As mentioned above, during the user's face and iris alignment process, the telescopic iris acquisition module and the wide-angle face acquisition camera continuously acquire the user's face and iris images. After initial alignment is completed, initial face and iris images are obtained, and the quality of the initial images is assessed to obtain images that meet the quality requirements.

[0052] In one embodiment of this application, evaluating the quality scores of an initial face image and an initial iris image includes: analyzing the sharpness of the initial iris image, determining whether there are obstructions in the initial iris image, and calculating the pupil center offset; calculating the quality score of the initial iris image based on the sharpness, the obstruction status of the obstruction, and the pupil center offset; and calculating the quality score of the initial face image based on the angle information, illumination information, and facial feature status information of the face in the initial face image.

[0053] For example, quality assessment includes iris image quality assessment and face image quality assessment. For iris image quality assessment, iris images acquired through a telescopic iris acquisition module are analyzed in real time, focusing on key indicators such as image sharpness, presence or absence of eyelashes or eyelid occlusion, and pupil center offset. A comprehensive quality score is calculated based on a preset assessment model, such as calculating the comprehensive quantitative index for iris image quality assessment (Iris Image Quality Score, or HQL for short).

[0054] For facial image quality assessment, facial images acquired through wide-angle facial capture cameras are analyzed simultaneously, mainly detecting parameters such as yaw angle, pitch angle, roll angle (i.e., the left and right tilt, up and down tilt and rotation angles of the face), illumination uniformity, and whether the eyes are closed.

[0055] Furthermore, in this embodiment, after evaluating the quality scores of the initial face image and the initial iris image, the method further includes: if the quality score of the initial face image or the initial iris image fails the evaluation, re-displaying the guidance prompt information for the image that failed the evaluation on the secondary display screen; and cyclically performing quality evaluation and adaptive adjustment based on the updated face image and iris image.

[0056] For example, based on the results of the two quality assessments mentioned above, specific and operable adjustment instructions are generated through the secondary display screen to provide users with precise real-time feedback. For instance, if an incorrect face angle is detected (yaw or pitch angle exceeding limits), the system prompts "Please face the screen directly"; if insufficient iris image clarity is detected, the system prompts "Please keep your head stable and adjust until the image is clear"; if uneven lighting is detected, the system prompts "Please adjust your face orientation to avoid direct sunlight or backlighting"; if the user's eyes are closed, the system prompts "Please keep your eyes open." Through this dynamic adaptive feedback mechanism, the user is guided to continuously adjust their posture and repeatedly perform image acquisition and quality assessment as described in the above embodiments until the quality assessment results meet the requirements. Therefore, this embodiment can ensure the quality of the acquired data from the source.

[0057] Step S104: If the quality scores of the initial face image and the initial iris image are both greater than the corresponding thresholds, capture the user's face image and iris image at the same time.

[0058] Specifically, this step involves the synchronous capture and data encapsulation of the final image. The control unit automatically triggers a high-speed synchronous capture mechanism only when both the overall quality score of the iris image and the quality assessment parameters of the face image exceed preset thresholds. This accurately acquires a pair of iris and face images that are strictly aligned in time and meet quality standards, ensuring the correlation and validity of the two biometric data.

[0059] Furthermore, the control unit can also jointly encapsulate the captured high-quality iris and face images with data collected by optional external devices.

[0060] In one embodiment of this application, after capturing the user's face image and iris image at the same time, the method further includes: packaging the captured face image and iris image with the user's identity information, fingerprint information, and high-definition photo data collected by an external device; and printing the packaged data using a thermal printer to output a business credential.

[0061] For example, the control unit packages the captured high-quality iris and facial images together with the user's identity text information (such as name and ID number) obtained through an ID card reader, as well as fingerprint information collected by an optional fingerprint scanner or a photograph taken by a document scanner, to form a complete user biometric and identity information data package. Finally, a business voucher containing, but not limited to, information such as the collection time, collection device number, data quality level, and specific content of the collected data is output through a thermal printer, completing a full collection process.

[0062] In summary, the multimodal biometric data acquisition method of this application significantly improves acquisition efficiency and user experience. Through graphical collaborative guidance and visual linkage design on the main and secondary displays, the previously fragmented multi-step processes of face and iris acquisition are integrated into a smooth, intelligent process. Users no longer need to repeatedly move their bodies to adjust distance, shortening the average acquisition time. The entire process requires no manual intervention; users can complete the operation entirely autonomously, reducing labor costs and avoiding operational discrepancies caused by human guidance, achieving an efficient and user-friendly self-service acquisition experience. Furthermore, this method significantly enhances data acquisition quality and reliability. By performing real-time dual-modal image quality assessment during acquisition and providing precise adaptive feedback based on the assessment results, it guides users to adjust their posture in a timely manner, preventing the entry of unqualified data into the database from the source. High-quality acquired data ensures the accuracy and stability of subsequent identification and authentication processes, meeting the application requirements for biometric data with strict standards. Moreover, this method improves the intelligence and integration level of the device, deeply combining hardware characteristics with software algorithms, fully leveraging the integrated advantages of the all-in-one machine. Meanwhile, the device supports the expansion of various optional peripherals and additional bio-information acquisition modules, and can be customized according to the needs of different application scenarios, thus broadening its application prospects in different fields.

[0063] Based on the above embodiments, in order to more clearly and intuitively describe the specific implementation process of the multimodal biometric data acquisition method of this application, a specific embodiment in a practical application is described below as an example. This embodiment takes a coal mine security access control scenario as an example.

[0064] In this embodiment, during the initial preparation phase, the equipment is deployed by installing the all-in-one machine next to the entrance / exit passage, connecting the power supply, and automatically loading the acquisition control software after the control unit starts up, completing the self-test and initialization of each hardware component. After the self-test passes, the main display screen shows a welcome interface, and the secondary display screen displays "Welcome to the multimodal biometric acquisition system, please prepare for acquisition," and the device enters standby mode.

[0065] Then, users can bring their ID cards to the all-in-one machine to have their biometric features collected for subsequent identity verification at entrances and exits.

[0066] During the acquisition phase, the multimodal biometric acquisition process includes the following steps. Step 1: Initial guidance and face reservation.

[0067] When a user approaches the all-in-one machine, the system detects the user's presence using a distance sensor and automatically switches from standby mode to data acquisition mode. The main display immediately shows a dynamic face positioning animation (the positioning frame is a green solid-line rectangle, and its size dynamically adjusts according to the detected face position), while the secondary display simultaneously shows the text prompt "Please face the main screen."

[0068] The wide-angle face capture camera is activated to capture real-time images of the user's face. The face detection algorithm identifies the user's face position and matches the face image with the positioning frame on the main display screen. The distance sensor simultaneously detects the distance between the user and the device and transmits the distance data to the control unit in real time.

[0069] If the user is more than 100cm away from the device (the upper limit of the effective distance), the secondary display screen will immediately show "Please move closer to the screen" and be accompanied by a soft voice prompt (the voice prompt can be emitted through the device's built-in speaker, and the volume can be adjusted); if the user is within the range of 30-100cm, and the wide-angle face capture camera successfully detects the face and matches the positioning frame, the system determines that the user has completed the face pre-positioning and proceeds to the next step.

[0070] The second step involves coordinating visual guidance and iris scanning alignment.

[0071] Once the face is pre-positioned, the main display screen automatically switches, the original face positioning frame disappears, and a red iris positioning circle (approximately 5cm in diameter) is displayed in the center of the screen. At the same time, the system uses facial key point detection technology to extract key point information such as the user's eye position and facial posture based on the face image acquired in real time by the wide-angle face capture camera. A virtual glasses frame icon that is linked to the user's facial posture is superimposed on the main display screen (the virtual glasses frame is a blue dotted frame, the size is adapted to the user's face proportion, and the position is adjusted synchronously in real time as the user's head moves).

[0072] The prompt on the secondary display screen has been updated to "Please move your head and align your eyes with the circle on the screen," while the speaker emits a voice prompt: "Please move your head so that your glasses frame aligns with the red circle."

[0073] Following the visual guidance on the main display and the text and voice prompts on the secondary display, the user slowly adjusts their head position (e.g., moving left and right, up and down, or slightly rotating). During this process, the virtual glasses frame constantly follows the user's facial posture. By observing the relative position of the virtual glasses frame and the iris positioning ring, the user gradually aligns their eyes with the optimal optical axis for iris acquisition. When the virtual glasses frame is fully fitted into the iris positioning ring, and the system detects that the user's eye position coincides with the physical optical axis of the iris acquisition module, the iris acquisition alignment is considered complete.

[0074] The third step is real-time quality assessment and adaptive feedback.

[0075] During the iris alignment process, the system initiates real-time quality assessment in parallel. For iris image quality assessment: the telescope-type iris acquisition module captures iris preview images in real time, and the control unit's quality assessment algorithm analyzes the image, calculating sharpness (based on the edge gradient operator, with a sharpness threshold of 0.6), eyelash / eyelid occlusion ratio (occlusion ratio threshold set to 10%), and pupil center offset (offset threshold set to 5 pixels). Based on the above indicators, a comprehensive quality score (HQL value, threshold set to 80 points) is calculated. For facial image quality assessment: The wide-angle facial capture camera simultaneously captures a facial preview image, and the algorithm detects the yaw angle (threshold set to ±15°), pitch angle (threshold set to ±10°), roll angle (threshold set to ±5°), illumination uniformity (determined based on image grayscale variance, threshold set to 0.3), and whether the eyes are closed (e.g., judged by the distance of key eye points).

[0076] If the assessment results show insufficient iris image clarity (e.g., HQL value < 80 points), the secondary display will prompt "Please keep your head stable and adjust until the image is clear," while the iris positioning circle on the main display will flash yellow; if the detected face yaw angle exceeds the standard (>15° or < -15°), the secondary display will prompt "Please face the screen directly and avoid tilting your face to the left or right," and the virtual glasses frame on the main display will display a red border; if the user's eyes are closed, the secondary display will prompt "Please open your eyes," accompanied by a voice reminder; if the lighting is uneven, the secondary display will prompt "Please adjust your face orientation to avoid direct sunlight or backlighting."

[0077] Based on the feedback information provided above, the user adjusts their head posture and position in real time, and the system continuously performs quality assessments until the quality indicators of both the iris image and the face image meet the preset thresholds.

[0078] The fourth step is to simultaneously capture and encapsulate the data.

[0079] When the system detects that the HQL value of the iris image is ≥80, and the yaw angle, pitch angle, and roll angle of the face image are all within the threshold range, the illumination uniformity meets the standard, and the user's eyes are open, the control unit immediately triggers the high-speed synchronous capture mechanism. The telescope-type iris acquisition module and the wide-angle face acquisition camera simultaneously capture images, respectively acquiring high-quality iris images and face images, ensuring that the two images are strictly aligned in time (time difference ≤10ms).

[0080] Users place their ID cards on the ID card reader, which quickly reads the user's name, ID number, gender, date of birth, and other identity information and transmits it to the control unit. If the user needs to have their fingerprints collected, they can press their finger on the fingerprint collector, which acquires the user's fingerprint image and transmits it to the control unit.

[0081] The control unit encapsulates iris images, facial images, ID card text information, and fingerprint images (optional) into a unified data package. The data packets are encrypted using the AES encryption algorithm to ensure the security of data transmission and storage. The encapsulated data packet includes additional information such as a data identifier (unique identification code), acquisition time, acquisition device number, and data quality level.

[0082] The control unit sends a print command to the thermal printer, which prints a business voucher containing information such as a successful data acquisition message, data identifier, acquisition time, and device number. The user then takes the voucher and their ID card, completing the entire data acquisition process.

[0083] In the subsequent processing stage, the encrypted data packets, once collected, can be transmitted to the backend management platform via the device's built-in network module (supporting both wired and wireless networks). The platform decrypts, stores, and manages the data packets for subsequent identity verification and authentication. Users can then quickly verify their identity by comparing their biometric features using the all-in-one machine or other authentication terminals when passing through access points.

[0084] To implement the above embodiments, this application also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multimodal biometric acquisition method as described in any one of the first aspects of the embodiments of this application.

[0085] It should be noted that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0086] Furthermore, in the description of this application, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0088] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0089] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0090] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this invention.

Claims

1. A method for acquiring multimodal biometric features, characterized in that, Includes the following steps: The system guides users to pre-position their faces by displaying a face positioning frame on the main screen and face positioning prompts on the secondary screen. After detecting that the user's face image is within the range of the face positioning frame, the main display screen is controlled to display the iris positioning circle and the virtual glasses frame that is linked to the user's facial posture, and the iris positioning prompt information is displayed through the secondary display screen to guide the user to move their head to align the virtual glasses frame with the iris positioning circle so that the user's eyes are located at the iris acquisition target position. The system uses a telescopic iris capture module and a wide-angle face capture camera to monitor the user's initial face image and initial iris image, and evaluates the quality scores of the initial face image and initial iris image. If the quality scores of the initial face image and the initial iris image are both greater than the corresponding thresholds, capture the user's face image and iris image at the same time.

2. The method according to claim 1, characterized in that, The face positioning frame displayed on the main screen and the face positioning prompts displayed on the secondary screen guide the user to perform face pre-positioning, including... The secondary display screen shows a prompt message guiding the user to face the main display screen. The wide-angle face capture camera monitors the user's facial information in real time, and the distance sensor detects the real-time distance between the user and the main display screen. If the real-time distance is greater than the distance threshold, the user is guided to adjust their position by displaying a position adjustment prompt on the secondary display screen until the user's face image is within the range of the face positioning frame.

3. The method according to claim 1, characterized in that, The control of the main display screen to display the iris positioning circle and the virtual glasses frame that is linked to the user's facial posture includes: The iris positioning circle is displayed in the center of the main display screen, and based on facial key point detection technology, the virtual glasses frame corresponding to the user's current facial posture is displayed on the main display screen; The step of displaying iris location information on the secondary display screen includes: The secondary display screen shows prompts guiding the user to move their head to align their eyes with the iris positioning circle, wherein at the iris acquisition target position, the physical light axis of the telescopic iris acquisition module corresponds to the position of the user's eyes.

4. The method according to claim 1, characterized in that, The evaluation of the quality scores of the initial face image and the initial iris image includes: Analyze the clarity of the initial iris image, determine whether there are any obstructions in the initial iris image, and calculate the pupil center offset; The quality score of the initial iris image is calculated based on the sharpness, the degree of occlusion by the obstruction, and the pupil center offset. The quality score of the initial face image is calculated based on the angle information, lighting information, and facial feature status information of the face in the initial face image.

5. The method according to claim 4, characterized in that, Following the evaluation of the quality scores of the initial face image and the initial iris image, the process further includes: If the quality score of the initial face image or initial iris image fails the evaluation, the guidance prompt information for the image that failed the evaluation will be re-displayed on the secondary display screen; Based on the updated face and iris images, quality assessment and adaptive adjustments are performed iteratively.

6. The system according to claim 1, characterized in that, After capturing the user's face and iris images at the same time, the method further includes: The captured facial and iris images are packaged together with the user's identity information, fingerprint information, and high-resolution photo data collected by external devices. Print the packaged data using a thermal printer to output business vouchers.

7. A multimodal biometric data acquisition device, characterized in that, include: The housing, and the main display screen, secondary display screen, telescopic iris scanning module, wide-angle face scanning camera, and control unit disposed within the housing; wherein... The main display screen is used to display animated images required for face localization and iris localization; The secondary display screen is used to display prompts that guide the user to perform face and iris localization. The telescope-type iris acquisition module is used to acquire the user's facial image; The wide-angle face capture camera is used to capture the user's iris image; The control unit establishes communication connections with each module in the housing, and the control unit is used to coordinate the control of each module to realize the multimodal biometric acquisition method as described in claim 1.

8. The all-in-one machine according to claim 7, characterized in that, Also includes: A distance sensor is used to detect the real-time distance between the user and the main display screen.

9. The all-in-one machine according to claim 7, characterized in that, Also includes: An ID card reader, wherein the ID card reader is used to collect the identity information of the user; A fingerprint collector, used to collect a user's fingerprint information; A high-resolution document scanner, used to collect high-definition photo data of items carried by a user; A thermal printer is used to print various data collected by the multimodal biometric data acquisition device and output business vouchers.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal biometric acquisition method as described in any one of claims 1-6.

Images