Augmented reality biometric enrollment process
Augmented reality-guided biometric enrollment simplifies and accelerates the process by virtually aligning hands for precise data capture, addressing the challenges of precision and hygiene in existing methods.
Patent Information
- Authority / Receiving Office
- FR · FR
- Patent Type
- Utility models
- Current Assignee / Owner
- BANKS & ACQUIRERS INT HLDG SAS
- Filing Date
- 2024-12-18
- Publication Date
- 2026-06-19
AI Technical Summary
The existing biometric enrollment process for palmar vein patterns is cumbersome and requires precise hand positioning, often involving physical guides that compromise hygiene and user experience, particularly during the enrollment phase.
A biometric enrollment method utilizing augmented reality to generate a virtual hand mask based on the user's hand shape, guiding optimal positioning without physical contact, and integrating it into a video stream for precise alignment and data capture.
Facilitates quick and hygienic enrollment by allowing all users to position their hands optimally, eliminating the need for physical guides and enhancing user experience.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Augmented reality biometric enrollment method
[0001] The invention relates to a biometric enrollment method. In particular, it relates to a biometric enrollment method involving the capture of the palmar vein pattern of a hand.
[0002] In this enrollment process, a user places the palm of their hand in front of a biometric capture device equipped with near-infrared sensors, in order to capture the vein pattern of the user's palm. Once the biometric data is thus captured, it is recorded and associated with the user in a database.
[0003] Subsequently, the user can undergo biometric identification. During the biometric identification process, the user also places the palm of their hand in front of a biometric capture device equipped with near-infrared sensors, in order to capture the vein pattern of that palm. The captured biometric data is compared to the data recorded in the database during enrollment, in order to identify the user.
[0004] When used for subsequent payment, the user associates their payment device, or a unique identifier of their payment device, with their identity during enrollment. Therefore, when making a payment, the user no longer needs to use their payment card or phone; they simply need to expose the palm of their hand to a biometric data capture device and undergo biometric identification. Such a project, linking biometric data from the vein pattern of a palm on the one hand, and a payment process on the other, is known as "Palm Vein Payment."
[0005] The biometric identification method using the vein pattern is particularly advantageous compared to other biometric identification methods. Indeed, in addition to the fact that the biometric data involved makes it very secure, it is carried out without contact between the hand and the capture device, which makes it particularly hygienic and therefore advantageous for the user.
[0006] The corresponding enrollment process is more cumbersome than the subsequent identification process. In particular, during enrollment, the user's hand must be presented to the sensors in a much more precise optimal position than during subsequent identification. Thus, during enrollment, the user must be precise and patient, which can lead to an uncomfortable experience and limit the adoption of this biometric identification method.
[0007] To address this need for precision during enrollment, a guide, in the form of a support placed in a predetermined position and at a predetermined distance from the sensors, is already known in the state of the art, allowing the user to place their fingers on it. In this way, the user is encouraged to expose the palm of their hand at a distance that approaches the appropriate position for enrollment.
[0008] However, this guide allows the user considerable freedom in positioning. Furthermore, it is designed for an "average" hand. Therefore, it cannot guarantee optimal positioning for every user.
[0009] That is why, in addition to this guide, text is displayed on a screen to indicate to the user how to better position their hand if necessary.
[0010] However, understanding and applying these instructions can also be difficult. Linguistic and iconographic confusions exist, such as confusion between requests to move one's hand "forward" or "upward".
[0011] Finally, the use of the guide involves contact between the hand and the guide, which goes against the "contactless" nature of this system and its hygienic advantage, and which risks hindering its adoption.
[0012] The invention aims to simplify and accelerate the biometric enrollment process relating to the capture of the palmar vein pattern of a hand, by a technique adapted to all hands and without contact between the hand and a support.
[0013] To this end, the invention relates to a biometric enrollment method comprising the implementation of the following steps:
[0014] - obtaining a video stream showing the vicinity of a capture device biometric data from one hand;
[0015] - a user presenting their hand to the capture device in order to perform biometric enrollment, the hand appearing within the video stream, obtaining a virtual mask of the hand from the video stream;
[0016] - generation of a virtual hand from the virtual mask, so that the the shape of the virtual hand corresponds to the shape of the user's hand;
[0017] - integration of the virtual hand generated in the video stream in a position indicating an optimal theoretical positioning of the user's hand, allowing optimal user enrollment, with respect to the capture device;
[0018] - display, on a screen visible to the user, of the video stream including the hand integrated virtual,
[0019] - once the user's hand positioning is validated, data capture biometric hand of the user,
[0020] The method includes, for the purpose of integrating the virtual hand, a step of measuring the distance between the user's hand and a camera that obtains the video stream, and further includes a step of moving the virtual hand or a modification of the appearance of the virtual hand in the video stream in order to encourage the user to move their hand.
[0021] Thus, the user's hand is virtually reproduced and integrated, using augmented reality, onto the screen in the optimal position where the real hand should be. The user viewing the screen can then superimpose their real hand onto the virtual hand. Positioning their hand is therefore facilitated, which speeds up enrollment and improves the user experience during enrollment. This method of guiding the user using augmented reality is suitable for all hands because it is the user's hand that is reproduced virtually. This technique also eliminates the need for physical contact between the hand and any support or device.
[0022] Furthermore, the integration of the hand in augmented reality does not only take into account the captured video stream but also the actual distance measured between the user's real hand and the camera, which makes it possible to improve the realism of the integration of the virtual hand, in particular to make the size of the virtual hand very precise depending on the position where it is integrated.
[0023] Furthermore, the user is prompted to move or manipulate their actual hand based on what is displayed on the screen. These virtual hand movements, corresponding to the user's hand, are more easily understood than textual instructions. Therefore, the hand repositioning is more effectively implemented.
[0024] Other optional features follow, taken alone or in combination.
[0025] Advantageously, the biometric enrollment method aims at the non-contact capture of biometric data concerning the palm of the user's hand, in particular the non-contact capture of data concerning veins of the palm of the user's hand.
[0026] Preferably, the method includes, once the enrollment has been validated, a step of associating the user's identity with a payment device and / or with a unique identifier of a payment device.
[0027] Thus, at a later payment, the user can simply perform biometric identification, and the payment can be made automatically, without the user having to handle a payment device, such as a phone or a payment card.
[0028] The invention also relates to a method of biometric identification of a user with a biometric identification device, in which a biometric enrollment of the user, allowing the biometric identification of this user, has been carried out beforehand in accordance with the biometric enrollment method described above.
[0029] The invention also relates to a payment method, comprising the implementation of the following steps:
[0030] - biometric identification of a physical buyer at a device biometric identification in accordance with the biometric identification process described above;
[0031] - if the buyer's biometric identification is validated, automatic payment of the buyer.
[0032] The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, lead the latter to implement the steps of the biometric enrollment process, the biometric identification process or the payment process as introduced above.
[0033] The invention also relates to a computer-readable recording medium comprising instructions which, when executed by a computer, lead the computer to implement the steps of the biometric enrollment process, the biometric identification process or the payment process as introduced above. Brief description of the figures
[0034] The invention will be better understood upon reading the following description, given solely by way of example and made with reference to the accompanying drawings in which:
[0035] [Fig-1] is a diagram of a biometric enrollment system according to a first embodiment of the invention;
[0036] [Fig.2] is a diagram of a biometric enrollment system according to a second method of implementing the invention;
[0037] [Fig.3] is a flowchart of a biometric enrollment process according to a mode of implementation of the invention; and
[0038] [Fig.4] is a flowchart of a biometric identification process according to a mode implementation of the invention.
[0039] [Fig.5] is a flowchart of a payment process according to a method of implementation work of the invention; and Detailed description
[0040] Figures 1 and 2 illustrate two embodiments of a biometric enrollment system. In both figures, these systems are shown schematically and not realistically. Furthermore, only the elements necessary for understanding the invention are represented.
[0041] In [Fig. 1], the biometric enrollment system 1 includes a device 2 for capturing biometric data from a user's hand 3. The system 1 also includes a camera 5 for obtaining a video stream including the vicinity of the capture device 2. The system 1 also includes a processing unit Computer Science 6. System 1 also includes a screen 7, visible to user 4, to display the video stream filmed by camera 5 and further including an augmented reality component according to method 100 described below. In this system 1, this screen 7 faces user 4 and is located in a plane distinct from the capture device 2. User 4 therefore looks either at their hand 3, which is to be exposed to the capture device 2, or at the screen 7, but cannot look at both at the same time.
[0042] The biometric data capture device 2 is an enrollment device designed for the contactless capture of biometric data concerning the veins of the user's palm 3. It is equipped with sensors that manipulate near-infrared waves. It is part of the "Palm Vein Payment" project, conducted in particular by the company INGENICO. Within the framework of this project, the vein pattern of the user 4 is "captured" by this device 2, that is to say, recorded in a database as millions of descriptor points of the vein pattern: this is the enrollment process. This technique makes it possible to uniquely identify the user 4 when they subsequently expose the palm of their hand 3 to a corresponding biometric identification device. The way in which this enrollment device 2 works, as well as the way in which the subsequent biometric identification works, are not the subject of the present invention.
[0043] The camera 5 is equipped with a LIDAR scanner 8 (for "Light Detection and Ranging") allowing the distance between the camera 5 and the hand 3 to be calculated. Alternatively, the scanner 8 is located in a device separate from the camera 5. In an alternative embodiment not shown, the system 1 could do without a LIDAR scanner.
[0044] The computer processing unit 6 is equipped with conventional automated means, in particular a processor 9. It is connected to the capture device 2, the camera 5, the LIDAR scanner 8, and the screen 7. It receives information from these components and controls them in accordance with the processes described below. To this end, the unit 6 implements a computer program 10 comprising instructions which, when the program 7 is executed by a computer, and specifically here by a processor such as the processor 9, cause the latter to implement the steps of the biometric enrollment process 100, the biometric identification process 200, or the payment process 300, as described below. To implement this program 10, the processing unit 6 includes a computer-readable recording medium 11 containing this program 10.In other words, this medium includes instructions which, when executed by a computer, or more precisely by a processor such as processor 9, lead it to implement the steps of the biometric enrollment process 100, the biometric identification process 200 or the payment process 300 as described below.
[0045] System 12 of [Fig. 2] comprises the same elements. It differs from system 1 of [Fig. 1] in that the screen 7 is located between the eyes of the user 4 and the capture device 2. Consequently, the hand 3 of the user 4, exposed to the capture device 2, is necessarily hidden by this screen 7. The user 4 only has to look in one direction, that of the screen 7.
[0046] We will now describe a biometric enrollment method 100, with reference to [Fig.3]. It is implemented via either of the two biometric enrollment systems 1 and 12 described above.
[0047] Step 101 is obtaining a video stream showing the vicinity of the biometric data capture device 2. This video stream is obtained by the camera 5.
[0048] The user presents their hand 3 to the capture device 2 in order to perform biometric enrollment. The hand 3 appears in the video stream.
[0049] Step 102 is obtaining a virtual mask of hand 3 from the video stream. To do this, the source code of computer program 10 provides a call to the BackgroundSubtractor function of the OpenCV library.
[0050] Step 103 is the generation of a virtual hand 13 from the virtual mask, such that the shape of the virtual hand 13 corresponds to the shape of the user's hand 3. To do this, the source code provides for the use of the ARFoundation package on the Unity platform. In particular, the code provides for the creation of a virtual object in Unity containing a two-dimensional sprite, directed towards the viewpoint of the camera 5, and in which the virtual hand 13 is inscribed.
[0051] Step 104 is the integration of the generated virtual hand 13 into the video stream in a position indicating a theoretically optimal positioning of the user's hand 4, allowing for optimal enrollment of the user 4 with respect to the capture device 2. To perform this step 104, it is first necessary to identify the optimal position in which to place the virtual hand 13 in the video stream. The corresponding actual position, that is, the optimal position with respect to the sensors of the device 2 for enrollment, is known or identified through testing. To identify it in the video stream, the source code of the computer program 7 provides for the use of an "anchor" from Google's ARCore SDK (Software Development Kit). Next, with a view to integrating the virtual hand 13, the code provides, via the LIDAR scanner, a step to measure the distance between the hand 3 of the user 4 and the camera 5 which obtains the video stream.In this way, the size of the virtual hand 13 corresponds as closely as possible to the size of the real hand 3 and does not depend on the distance between the latter and the camera 5. In an unillustrated implementation mode, without a LIDAR scanner, it remains possible to integrate the virtual hand 13 into the optimal position, the size of the virtual hand 13 then being determined solely based on data from the video stream.
[0052] In conclusion to this step 104, the Unity sprite containing the virtual hand 13 is integrated into the video stream at the theoretically optimal position to allow enrollment. The shape and size of the virtual hand 10 correspond to those that the real hand 3 would have in the same position, making this integration particularly realistic.
[0053] Other techniques, in particular the use of other computer development platforms, can be implemented to achieve this integration in augmented reality.
[0054] Step 105 is the display, on the screen 7 visible to user 4, of the video stream including the integrated virtual hand 13. In other words, the virtual hand 13 is displayed in augmented reality on the screen 7. User 4 then simply places their hand 3 in the position of the virtual hand 13 to complete the enrollment. User 4 thus completes the enrollment quickly and easily, without any physical contact with a device or guide.
[0055] Step 106 is optional. It is only implemented when the user 4 does not position their real hand 3 in the optimal position despite the integration of the virtual hand 13. This step 106 involves moving the virtual hand 13 or modifying its appearance in the video stream to encourage the user 4 to move their hand 3. Thus, the virtual hand 13 moves, or deforms, in a given direction to prompt the user 4 to instinctively move it closer to the optimal position for enrollment. It can also involve making the virtual hand 13 translucent so that the user 4 can observe their real hand 3 positioned below the virtual hand 13 on the screen 7.
[0056] Step 107 is, once a positioning of the hand 3 of user 4 has been validated, the capture of the biometric data of the hand 3 of user 4. The enrollment is then completed.
[0057] Step 108 involves associating the identity of user 4 with a payment device and / or a unique identifier of a payment device. Thus, system 1 allows user 4, via screen 7 and other unillustrated means of interaction, to provide their payment information to the enrollment system 1. In particular, it allows for the provision of a unique identifier for a payment method; this is referred to as tokenization of the payment device. The user could also associate their identity with an identifier of a store loyalty program or a cryptocurrency wallet, which, within the scope of this invention, should be understood as payment devices. In this way, their identity is associated with their payment methods. They will only need to identify themselves by displaying the palm of their hand 3 to make a subsequent payment.
[0058] It follows from the process 100 that the automated means of the processing unit 6, such as the processor 9, are configured to:
[0059] - obtain a virtual mask of hand 3 from the video stream;
[0060] - generate a virtual hand 13 from the virtual mask, so that the shape of the virtual hand 13 corresponds to the shape of the hand 3 of the user 4;
[0061] - integrate the virtual hand 13 generated into the video stream, in a position indicating an optimal theoretical positioning of the user's hand 4, allowing optimal enrollment of the user 4, with respect to the capture device 2;
[0062] - once the positioning of user 4's hand 3 has been validated, capture user's hand biometric data 3.
[0063] The method 200 of [Fig. 4] is a biometric identification method according to an embodiment of the invention. It follows the implementation of the biometric enrollment method 100. The only step of this method 200 is therefore the identification, using a biometric data capture device, of the biometric data of the vein pattern of the palm of the user's hand 3. The user having previously enrolled using system 1, the data captured during identification are compared to the data recorded in the database during enrollment, and the identification is validated when the user is identified in the database.
[0064] Method 300 of [Fig. 5] is a payment method. It follows the implementation of biometric enrollment method 100.
[0065] Step 301 of this process biometric identification of user 4, who in this case has the role of buyer, from a biometric identification device, in accordance with process 200.
[0066] Step 302, if the buyer's biometric identification is validated, involves the automatic payment of the buyer. Thus, the user does not have to handle their payment card, smartphone, or other payment device. They simply have to expose the palm of their hand to a biometric data capture device. The data is then compared to the data recorded in the database during enrollment in process 100, in order to identify the user. Furthermore, the payment is processed automatically since their payment device was associated with their biometric identity in step 108 of enrollment process 100.
[0067] The invention is not limited to the embodiments shown and other embodiments will be obvious to a person skilled in the art. List of references
[0068] 1: Biometric enrollment system 2: Biometric data capture device 3: real hand
[0069]
[0070]
[0071]
[0072]
[0073]
[0074]
[0075]
[0076]
[0077]
[0078]
[0079]
[0080]
[0081] 4: user 5: camera 6: computer processing unit 7: screen 8: LIDAR scanner 9: processor 10: computer program 11: data carrier 12: biometric enrollment system 13: virtual hand 100: biometric enrollment method 200: biometric identification method 300: payment method
Claims
Demands
1. A biometric enrollment method (100), characterized in that it comprises the implementation of the following steps: - obtaining (101) a video stream showing the vicinity of a biometric data capture device (2) of a hand (3); - a user (4) presenting their hand to the capture device (2) in order to carry out biometric enrollment, the hand (3) appearing within the video stream, obtaining (102) a virtual mask of the hand (3) from the video stream; - generating (103) a virtual hand (13) from the virtual mask, such that the shape of the virtual hand (13) corresponds to the shape of the hand (3) of the user (4); - integration (104) of the virtual hand (13) generated in the video stream in a position indicating an optimal theoretical positioning of the hand (3) of the user (4), allowing optimal enrollment of the user (4), vis-à-vis the capture device (2);- display (105), on a screen (7) visible to the user (4), of the video stream including the integrated virtual hand (13), - once a positioning of the hand (3) of the user (4) has been validated, capture (107) of the biometric data of the user's hand, the method comprising, for the purpose of the integration (104) of the virtual hand, a step of measuring the distance between the hand (3) of the user (4) and a camera (5) obtaining the video stream, and further comprising a step (106) of moving the virtual hand (13) or a modification of the appearance of the virtual hand (13) in the video stream so as to encourage the user (4) to move their hand (3).;
2. Method (100) according to claim 2, wherein the biometric enrollment method aims at the non-contact capture of biometric data concerning a palm of the user's (4) hand (3), in particular the non-contact capture of data concerning veins of the palm of the user's (4) hand (3).
3. A method (100) according to any one of the preceding claims, comprising, once enrollment has been validated, a step of associating the identity of the user (4) with a payment device and / or with a unique identifier of a payment device.
4. A biometric identification method (200) of a user (4) with a biometric identification device, characterized in that a biometric enrollment of the user (4), enabling the biometric identification of this user (4), has been carried out beforehand in accordance with the biometric enrollment method (100) according to any one of claims 1 to 3.
5. Payment method (300), comprising the implementation of the following steps: - biometric identification (301) of a physical buyer (4) with a biometric identification device in accordance with the biometric identification method (200) of the preceding claim; - if the biometric identification of the buyer is validated, automatic payment (302) of the buyer (4).
6. Computer program (10) comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the biometric enrollment process (100) according to any one of claims 1 to 3, the biometric identification process (200) according to claim 4 or the payment process (300) according to claim 5.
7. Computer-readable recording medium (11) comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the biometric enrollment method (100) according to any one of claims 1 to 3, the biometric identification method (200) according to claim 4 or the payment method (300) according to claim 5.