Augmented reality biometric enrolment method

Abstract

The invention relates to a biometric enrolment method comprising the implementation of the following steps: - obtaining a video stream showing the vicinity of a capture device (2) for capturing biometric data from a hand (3); - when a user (4) displays their hand to the capture device (2), obtaining a virtual mask of the hand (3) from the video stream; - generating a virtual hand (13) from the virtual mask; - integrating the generated virtual hand (13) into the video stream in a position indicating an optimal theoretical position of the hand (3) of the user (4); - displaying the video stream comprising the integrated virtual hand (13) on a screen (7) visible to the user (4); - once the position of the hand (3) of the user (4) has been validated, capturing the biometric data from the hand of the user.

Description

Augmented reality biometric enrollment process

[0001] The invention relates to a biometric enrollment method. In particular, it relates to a biometric enrollment method involving the capture of the palmar venous pattern of a hand.

[0002] In this enrollment process, a user places the palm of their hand in front of a biometric capture device equipped with near-infrared sensors, in order to capture the vein pattern of the user's palm. Once the biometric data is captured, it is stored and associated with the user in a database.

[0003] Subsequently, the user can proceed with biometric identification. During the biometric identification process, the user also places the palm of their hand in front of a biometric capture device equipped with near-infrared sensors, in order to capture the vein pattern of the palm of this hand. The captured biometric data is compared to the data stored in the database during enrollment, in order to identify the user.

[0004] Used for subsequent payment, the user associates their payment device or a unique identifier of their payment device with their identity during enrollment. Therefore, when making a payment, the user no longer needs to use their payment card or phone to pay, they simply expose the palm of their hand to a biometric data capture device and proceed with their biometric identification. Such a project associating the biometric data of the vein pattern of a palm on the one hand, and a payment process on the other hand, is known as "PalmVeinPayment".

[0005] The biometric identification method using the vein pattern is particularly advantageous compared to other biometric identification methods. In fact, in addition to the biometric data involved making it very secure, it is carried out without contact between the hand and the capture device, which makes it particularly hygienic and therefore advantageous for the user.

[0006] The corresponding enrollment process is more tedious than the subsequent identification process. In particular, it is necessary that, during enrollment, the user's hand be presented to the sensors in a much more precise optimal position than during subsequent identification. Thus, during enrollment, the user must be precise and patient, which can generate an uncomfortable experience and limit the adoption of this biometric identification process.

[0007] To overcome this need for precision during enrollment, a guide is already known in the prior art, taking the form of a support placed in a predetermined position and distance from the sensors, allowing the user to place his fingers there. In this way, the user is encouraged to expose the palm of his hand at a distance which approaches the appropriate position for enrollment.

[0008] However, this guide allows the user a great deal of freedom in positioning. Furthermore, it is adapted to a so-called "average" hand. It is therefore not sufficient to ensure optimal positioning for every user.

[0009] Therefore, in addition to this guide, text is displayed on a screen to indicate to the user how to better position his hand if necessary.

[0010] However, understanding and applying these instructions can also be difficult. Linguistic and iconographic confusions exist, such as confusion between requests to move one's hand "forward" or "upward."

[0011] Finally, the use of the guide involves contact between the hand and the guide, which goes against the "contactless" nature of this system and its hygienic advantage, and which risks slowing its adoption.

[0012] The invention aims to simplify and accelerate the biometric enrollment process involving the capture of the palmar venous pattern of a hand, using a technique suitable for all hands and without contact between the hand and a support.

[0013] To this end, the invention relates to a biometric enrollment method comprising the implementation of the following steps:

[0014] - obtaining a video stream showing the vicinity of a hand biometric data capture device;

[0015] - a user presenting his hand to the capture device in order to carry out biometric enrollment, the hand appearing within the video stream, obtaining a virtual mask of the hand from the video stream;

[0016] - generation of a virtual hand from the virtual mask, so that the shape of the virtual hand corresponds to the shape of the user's hand;

[0017] - integration of the generated virtual hand into the video stream in a position indicating an optimal theoretical positioning of the user's hand, allowing optimal enrollment of the user, with respect to the capture device;

[0018] - display, on a screen visible to the user, of the video stream including the integrated virtual hand,

[0019] - once the positioning of the user's hand is validated, captures biometric data of the user's hand.

[0020] Thus, the user's hand is virtually reproduced and integrated, in augmented reality, on the screen, in the optimal position where the real hand should be. The user looking at the screen can then superimpose his real hand on the virtual hand. The positioning of his hand is therefore facilitated, which accelerates enrollment and improves the user experience during it. This way of guiding the user by augmented reality is suitable for all hands because it is the user's hand that is reproduced in virtual form. This technique also makes it possible to do without physical contact between the hand and any support or device.

[0021] Other optional features follow, taken alone or in combination.

[0022] Advantageously, the method comprises, with a view to integrating the virtual hand, a step of measuring the distance between the user's hand and a camera obtaining the video stream.

[0023] Thus, the integration of the hand in augmented reality does not only take into account the captured video stream but also the actual distance measured between the user's real hand and the camera, which makes it possible to improve the realism of the integration of the virtual hand, in particular to make the size of the virtual hand very precise depending on the position where it is integrated.

[0024] Preferably, the method comprises a step of moving the virtual hand or modifying the appearance of the virtual hand in the video stream so as to encourage the user to move his hand.

[0025] Thus, the user is prompted to move or reposition their real hand based on what is displayed on the screen. However, these virtual modifications of a hand corresponding to the user's hand are more easily understood than text instructions. Hand repositioning is therefore better applied.

[0026] Advantageously, the biometric enrollment method aims at the contactless capture of biometric data concerning the palm of the user's hand, in particular the contactless capture of data concerning veins in the palm of the user's hand.

[0027] Preferably, the method comprises, once the enrollment has been validated, a step of associating the identity of the user with a payment device and / or with a unique identifier of a payment device.

[0028] Thus, during a subsequent payment, the user can simply carry out biometric identification, and the payment can be made automatically, without the user having to handle a payment device, such as a telephone or a payment card.

[0029] The invention also relates to a method for biometric identification of a user with a biometric identification device, in which a biometric enrollment of the user, allowing the biometric identification of this user, has been carried out beforehand in accordance with the biometric enrollment method described above.

[0030] The invention also relates to a payment method, comprising the implementation of the following steps:

[0031] - biometric identification of a physical buyer using a biometric identification device in accordance with the biometric identification process described above;

[0032] - if the buyer's biometric identification is validated, automatic payment by the buyer.

[0033] The invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the latter to implement the steps of the biometric enrollment method, the biometric identification method or the payment method as introduced above.

[0034] The invention also relates to a computer-readable recording medium comprising instructions which, when executed by a computer, cause the latter to implement the steps of the biometric enrollment method, the biometric identification method or the payment method as introduced above.

[0035] The invention also relates to a biometric enrollment system comprising:

[0036] - a device for capturing biometric data from a user's hand;

[0037] - a camera for obtaining a video stream including the vicinity of the capture device;

[0038] - automated means configured to:

[0039] ** get a virtual hand mask from the video stream;

[0040] ** generate a virtual hand from the virtual mask, so that the shape of the virtual hand matches the shape of the user's hand;

[0041] ** integrate the generated virtual hand into the video stream, in a position indicating an optimal theoretical positioning of the user's hand, allowing optimal enrollment of the user, with respect to the capture device;

[0042] ** once a user's hand positioning is validated, capture biometric data of the user's hand;

[0043] - a screen, visible to the user, to display the video stream including the integrated virtual hand. Brief description of the figures

[0044] The invention will be better understood on reading the following description, given solely by way of example and with reference to the appended drawings in which:

[0045] is a diagram of a biometric enrollment system according to a first embodiment of the invention;

[0046] is a diagram of a biometric enrollment system according to a second embodiment of the invention;

[0047] is a flowchart of a biometric enrollment method according to an embodiment of the invention; and

[0048] is a flowchart of a biometric identification method according to an embodiment of the invention.

[0049] is a flowchart of a payment method according to an embodiment of the invention; and Detailed description

[0050] Figures 1 and 2 show two embodiments of a biometric enrollment system. In both figures, these systems are illustrated schematically and not realistically. Furthermore, only the elements necessary for understanding the invention are shown.

[0051] On the, the biometric enrollment system 1 comprises a device 2 for capturing biometric data from a hand 3 of a user 4. The system 1 also comprises a camera 5 for obtaining a video stream comprising the vicinity of the capture device 2. The system 1 also comprises a computer processing unit 6. The system 1 also comprises a screen 7 visible to the user 4, for displaying the video stream comprising filmed by the camera 5 and further comprising an augmented reality portion according to the method 100 described below. In this system 1, this screen 7 faces the user 4 and is located in a plane separate from the capture device 2. The user 4 therefore looks either at his hand 3 to be exposed to the capture device 2, or at the screen 7, but he cannot look at both at the same time.

[0052] The biometric data capture device 2 is an enrollment device for the contactless capture of biometric data concerning veins in the palm of the user's hand 3. For this purpose, it is equipped with sensors that manipulate near-infrared waves. It is part of the "PalmVeinPayment" project led in particular by the company INGENICO. As part of this project, the venous pattern of the user 4 is "captured" by this device 2, i.e. recorded in a database in the form of millions of descriptor points of the venous pattern: this is the enrollment. This technique makes it possible to uniquely identify the user 4 when he subsequently exposes the palm of his hand 3 to a corresponding biometric identification device. The manner in which this enrollment device 2 operates, as well as the manner in which the subsequent biometric identification operates, are not the subject of the present invention.

[0053] The camera 5 is equipped with a LIDAR scanner 8 (for “LightDetectionAndRanging”) making it possible to calculate the distance between the camera 5 and the hand 3. Alternatively, the scanner 8 is located in a device separate from the camera 5. In an alternative embodiment not illustrated, the system 1 could do without a LIDAR scanner.

[0054] The computer processing unit 6 is provided with conventional automated means, in particular a processor 9. It is connected to the capture device 2, to the camera 5, to the LIDAR scanner 8, and to the screen 7. It receives information from these components and controls these components in accordance with the methods described below. For this purpose, the unit 6 implements a computer program 10 comprising instructions which, when the program 7 is executed by a computer, and specifically here by a processor such as the processor 9, lead the latter to implement the steps of the biometric enrollment method 100, the biometric identification method 200 or the payment method 300 as they will be described below. To implement this program 10, the processing unit 6 comprises a computer-readable recording medium 11 comprising this program 10.In other words, this medium comprises instructions which, when executed by a computer, or more precisely by a processor such as the processor 9, lead the latter to implement the steps of the biometric enrollment method 100, of the biometric identification method 200 or of the payment method 300 as described below.

[0055] The system 12 of the comprises the same elements. It differs from the system 1 of the in that the screen 7 is located between the eyes of the user 4 and the capture device 2. Therefore, the hand 3 of the user 4 exposed to the capture device 2 is necessarily hidden by this screen 7. The user 4 only has to look in one direction, that of the screen 7.

[0056] A biometric enrollment method 100 will now be described, with reference to the. It is implemented via any of the two biometric enrollment systems 1 and 12 described above.

[0057] Step 101 is obtaining a video stream showing the vicinity of the biometric data capture device 2. This video stream is obtained by the camera 5.

[0058] The user presents his hand 3 to the capture device 2 in order to carry out biometric enrollment. Hand 3 appears in the video stream.

[0059] Step 102 is to obtain a virtual mask of hand 3 from the video stream. To do this, the source code of computer program 10 provides a call to the BackgroundSubtractor function of the OpenCV library.

[0060] Step 103 is the generation of a virtual hand 13 from the virtual mask, so that the shape of the virtual hand 13 corresponds to the shape of the user's hand 3. To do this, the source code plans to make use of the ARFoundation package on the Unity platform. In particular, the code plans to create a virtual object, in Unity, containing a two-dimensional sprite, directed towards the viewpoint of the camera 5, and in which the virtual hand 13 is written.

[0061] Step 104 is the integration of the virtual hand 13 generated in the video stream in a position indicating an optimal theoretical positioning of the hand of the user 4, allowing an optimal enrollment of the user 4, with respect to the capture device 2. To carry out this step 104, it is first necessary to identify the optimal position where to place the virtual hand 13 in the video stream. The corresponding real position, that is to say the optimal position with respect to the sensors of the device 2 to carry out the enrollment, is known or identified by tests. To identify it in the video stream, the source code of the computer program 7 provides for the use of an “anchor” from the “SDK” (for “SoftwareDevelopmentKit”)ARCore, from Google. Then, with a view to integrating the virtual hand 13, the code provides, via the LIDAR scanner, a step of measuring the distance between the hand 3 of the user 4 and the camera 5 obtaining the video stream.In this way, the size of the virtual hand 13 corresponds as closely as possible to the size of the real hand 3 and does not depend on the distance between the latter and the camera 5. In an implementation mode not illustrated, without a LIDAR scanner, it remains possible to proceed with the integration of the virtual hand 13 at the optimal position, the size of the virtual hand 13 then being determined solely on the basis of the data from the video stream.

[0062] At the end of this step 104, the Unity “sprite” where the virtual hand 13 is registered is integrated into the video stream at the optimal theoretical position to enable enrollment. The shape and size of the virtual hand 10 correspond to those that the real hand 3 would have in the same position, making this integration particularly realistic.

[0063] Other techniques, in particular the use of other IT development platforms, can be implemented to achieve this integration in augmented reality.

[0064] Step 105 is the display, on the screen 7 visible to the user 4, of the video stream comprising the integrated virtual hand 13. In other words, the virtual hand 13 is displayed in augmented reality on the screen 7. From then on, the user 4 only has to place his hand 3 at the position of the virtual hand 13 so as to carry out the enrollment. The user 4 therefore carries out the enrollment quickly and simply, and without having any physical contact with any device or guide.

[0065] Step 106 is optional. It is only implemented when the user 4 does not place his real hand 3 in the optimal position despite the integration of the virtual hand 13. This step 106 is in fact the movement of the virtual hand 13 or a modification of the appearance of the virtual hand 13 in the video stream so as to encourage the user 4 to move his hand 3. Thus, the virtual hand 13 moves, or deforms, in a given direction so as to encourage the user 4 to perform a movement instinctively, bringing him closer to the optimal position for enrollment. It may also involve making the virtual hand 13 translucent so that the user 4 can observe, on the screen 7, his real hand 3 placed below the virtual hand 13.

[0066] Step 107 is, once a positioning of the hand 3 of the user 4 has been validated, the capture of the biometric data of the hand 3 of the user 4. The enrollment is then completed.

[0067] Step 108 is the association of the identity of the user 4 with a payment device and / or with a unique identifier of a payment device. Thus, the system 1 allows the user 4, via the screen 7 and other means of interaction not illustrated, to provide his payment information to the enrollment system 1. In particular, it allows to provide a unique identifier of a means of payment, this is referred to as tokenization of the payment device. The user could also associate his identity with an identifier of a store kitty or a cryptocurrency wallet, which must be understood in the context of this invention as payment devices. In this way, his identity is associated with his means of payment. He will only have to identify himself by exposing the palm of his hand 3 to make a subsequent payment.

[0068] It results from the method 100 that the automated means of the processing unit 6, such as the processor 9, are configured to:

[0069] - obtain a virtual mask of hand 3 from the video stream;

[0070] - generating a virtual hand 13 from the virtual mask, so that the shape of the virtual hand 13 corresponds to the shape of the hand 3 of the user 4;

[0071] - integrate the virtual hand 13 generated into the video stream, in a position indicating an optimal theoretical positioning of the hand of the user 4, allowing optimal enrollment of the user 4, with respect to the capture device 2;

[0072] - once a positioning of the hand 3 of the user 4 has been validated, capture biometric data of the hand 3 of the user.

[0073] The method 200 of the is a biometric identification method according to an embodiment of the invention. It follows the implementation of the biometric enrollment method 100. The only step of this method 200 is therefore the identification, with a biometric identification data capture device, of the biometric data of the venous pattern of the palm of the hand 3 of the user 4. The latter having previously enrolled using the system 1, the data captured during the identification are compared with the data recorded in the database during the enrollment, and the identification is validated when the user is identified in the database.

[0074] The method 300 is a payment method. It follows the implementation of the biometric enrollment method 100.

[0075] Step 301 of this method biometric identification of the user 4, who in this case has the role of buyer, with a biometric identification device, in accordance with the method 200.

[0076] Step 302 is, if the buyer's biometric identification is validated, the buyer's automatic payment. Thus, the user does not have to handle his payment card, his smartphone or another payment device. He only has to expose the palm of his hand to a biometric data capture device. The data is then compared to the data recorded in the database during the enrollment of the method 100, in order to identify the user 4. Furthermore, the payment is made automatically since his payment device was associated with his biometric identity in step 108 of the enrollment method 100.

[0077] The invention is not limited to the embodiments presented and other embodiments will become apparent to those skilled in the art. List of references

[0078] 1: biometric enrollment system2: biometric data capture device3: real hand

[0079] 4: user

[0080] 5: camera

[0081] 6: computer processing unit

[0082] 7: screen

[0083] 8: LIDAR scanner

[0084] 9: processor

[0085] 10: computer program

[0086] 11: data support

[0087] 12: Biometric enrollment system

[0088] 13: virtual hand

[0089] 100: biometric enrollment process

[0090] 200: biometric identification process

[0091] 300: payment method

Claims

A biometric enrollment method (100), characterized in that it comprises implementing the following steps: - obtaining (101) a video stream showing the vicinity of a capture device (2) for capturing biometric data of a hand (3); - a user (4) presenting his hand to the capture device (2) in order to carry out biometric enrollment, the hand (3) appearing within the video stream, obtaining (102) a virtual mask of the hand (3) from the video stream; - generating (103) a virtual hand (13) from the virtual mask, so that the shape of the virtual hand (13) corresponds to the shape of the hand (3) of the user (4); - integrating (104) the virtual hand (13) generated in the video stream in a position indicating an optimal theoretical positioning of the hand (3) of the user (4), allowing optimal enrollment of the user (4), with respect to the capture device (2);- display (105), on a screen (7) visible to the user (4), of the video stream comprising the integrated virtual hand (13), - once a positioning of the hand (3) of the user (4) has been validated, capture (107) of the biometric data of the user's hand.; Method (100) according to the preceding claim, comprising, with a view to the integration (104) of the virtual hand, a step of measuring the distance between the hand (3) of the user (4) and a camera (5) obtaining the video stream. Method (100) according to any one of the preceding claims, comprising a step (106) of moving the virtual hand (13) or modifying the appearance of the virtual hand (13) in the video stream so as to encourage the user (4) to move his hand (3). Method (100) according to any one of the preceding claims, wherein the biometric enrollment method aims at the contactless capture of biometric data concerning a palm of the hand (3) of the user (4), in particular the contactless capture of data concerning veins of the palm of the hand (3) of the user (4). Method (100) according to any one of the preceding claims, comprising, once the enrollment has been validated, a step of associating (108) the identity of the user (4) with a payment device and / or with a unique identifier of a payment device. Method for biometric identification (200) of a user (4) with a biometric identification device, characterized in that a biometric enrollment of the user (4), allowing the biometric identification of this user (4), has been carried out beforehand in accordance with the biometric enrollment method (100) according to any one of claims 1 to 5. Payment method (300), comprising the implementation of the following steps: - biometric identification (301) of a physical buyer (4) with a biometric identification device in accordance with the biometric identification method (200) of the preceding claim; - if the biometric identification of the buyer is validated, automatic payment (302) of the buyer (4). A computer program (10) comprising instructions which, when the program is executed by a computer, cause the computer to implement the steps of the biometric enrollment method (100) according to any one of claims 1 to 5, of the biometric identification method (200) according to claim 6 or of the payment method (300) according to claim 7. A computer-readable recording medium (11) comprising instructions which, when executed by a computer, cause the computer to implement the steps of the biometric enrollment method (100) according to any one of claims 1 to 5, of the biometric identification method (200) according to claim 6 or of the payment method (300) according to claim 7. Biometric enrollment system (1; 12), characterized in that it comprises:- a device (2) for capturing biometric data from a hand (3) of a user (4);- a camera (5) for obtaining a video stream comprising the vicinity of the capture device (2);- automated means (6) configured to:** obtain a virtual mask of the hand (3) from the video stream;** generate a virtual hand (13) from the virtual mask, so that the shape of the virtual hand (13) corresponds to the shape of the hand (3) of the user (4);** integrate the generated virtual hand (13) into the video stream, in a position indicating an optimal theoretical positioning of the hand (3) of the user (4), allowing an optimal enrollment of the user (4), with respect to the capture device (2);** once a positioning of the hand (3) of the user (4) has been validated, capture biometric data from the hand (3) of the user (4);- a screen (7), visible to the user (4), for displaying the video stream including the integrated virtual hand (13).;

Images