Multi-Factor Authentication for Virtual Reality

Abstract

To provide a multi-factor authentication method, a system, and a device suitable for a virtual reality (VR) system using a head-mounted display (HMD).SOLUTION: A process 500 for performing authentication for a virtual reality (VR) system using a plurality of authentication factors comprises the steps of: collecting, by selecting a series of data including numbers, letters or images on a virtual keyboard displayed on a display of the VR system by a user, a password for a first factor of a plurality of authentication factors entered on a virtual keyboard; and collecting a biometric fingerprint for a second factor of the plurality of authentication factors from movement of the VR system.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to virtual reality (VR) systems, and more particularly to multi-factor authentication for VR systems. [Background technology]

[0002] As virtual reality (VR) becomes more prevalent, some VR applications may be able to access sensitive or confidential information in the future, and therefore more robust authentication may be required in the future. Summary of the Invention [Problem to be solved by the invention]

[0003] Currently, most VR systems input digital information using a virtual keyboard displayed within a head-mounted display (HMD). In these systems, users activate the keyboard using either hand gesture recognition with a data glove or a wand that points to the displayed keys. However, these types of user interfaces are complex and cumbersome. [Means for solving the problem]

[0004] The present disclosure provides multi-factor authentication suitable for VR systems that use head-mounted displays (HMDs).

[0005] In one implementation, a method for performing authentication for a VR system using multiple authentication factors is disclosed, the method including collecting a password for a first one of the multiple authentication factors entered into a virtual keyboard by a user selecting a sequence of data including numbers, letters, or images on the virtual keyboard displayed on a display of the VR system, and collecting a biometric fingerprint for a second one of the multiple authentication factors from the movement of the VR system.

[0006] In one implementation, the VR system is a head mounted display (HMD). In one implementation, the method further includes displaying a virtual keyboard on the hemisphere at a fixed location within the hemisphere. In one implementation, the virtual keyboard is arranged as a random display of at least 10 keys on the hemisphere. In one implementation, the password is entered by gazing in the direction of a button on the virtual keyboard and selecting a button. In one implementation, the selected button is confirmed by making a predefined head gesture of approval. In one implementation, the selected button is deselected by making a predefined head gesture of deselection. In one implementation, a biometric fingerprint is collected from the movement of the VR system as reflected in the predefined head gesture of approval and the predefined head gesture of deselection. In one implementation, collecting the biometric fingerprint includes measuring accelerations of the predefined head gesture of approval and the predefined head gesture of deselection, and normalizing the accelerations. In one implementation, the method further includes comparing the collected biometric fingerprint with fingerprints in a fingerprint database and identifying the user when a fingerprint match occurs. In one implementation, a fingerprint match means that the similarity of the fingerprint match between the collected biometric fingerprint and the reference fingerprint is within a specified limit.

[0007] In another implementation, a system for performing authentication for a VR system using multiple authentication factors is disclosed, the system includes a first authentication collector element configured to collect a password entered into a virtual keyboard by a user selecting a sequence of data including numbers or images on the virtual keyboard displayed on a display of the VR system, and a second authentication collector element configured to collect a biometric fingerprint from the movement of the VR system.

[0008] In one implementation, the VR system is a head mounted display (HMD). In one implementation, the system further includes a projector configured to display a virtual keyboard on the hemisphere at a fixed position within the hemisphere. In one implementation, the virtual keyboard is arranged as a random display of at least 10 keys on the hemisphere. In one implementation, the second authentication collector component is configured to collect a biometric fingerprint from a movement of the VR system as reflected in a predefined head gesture. In one implementation, the second authentication collector component collects the biometric fingerprint by measuring an acceleration of the predefined head gesture. In one implementation, the system further includes a comparator configured to compare the collected biometric fingerprint with fingerprints in a fingerprint database, and an identifier configured to identify the user when a fingerprint match occurs.

[0009] In yet another implementation, an apparatus for performing authentication for a VR system using multiple authentication factors is disclosed, the apparatus including first collecting means for collecting a password for a first one of the multiple authentication factors entered into a virtual keyboard by a user selecting a sequence of data including numbers or images on the virtual keyboard displayed on a display of the VR system, and second collecting means for collecting a biometric fingerprint for a second one of the multiple authentication factors from a movement of the VR system.

[0010] In one implementation, the second collection means collects the biometric fingerprint from movements of the VR system as reflected in a predefined head gesture. In one implementation, the second collection means collects the biometric fingerprint by measuring an acceleration of the predefined head gesture.

[0011] Other features and advantages will become apparent from the present specification, which illustrates by way of example embodiments of the disclosure.

[0012] Details of the present disclosure, both as to its structure and operation, can be gleaned in part by studying the accompanying drawings, in which like parts are designated with like reference numerals. [Brief description of the drawings]

[0013] [Figure 1] 1 is a layout of a virtual keypad displayed on a sphere according to one implementation of the present disclosure. [Diagram 2] 1 is a layout of a virtual keypad displayed on a sphere but projected in polar coordinate form, according to one implementation of the present disclosure. [Diagram 3] FIG. 11 is a flow diagram illustrating a process of detecting a phase according to one implementation of the present disclosure. [Figure 4] FIG. 1 is a flow diagram illustrating a process for detecting an approval head gesture according to one implementation of the present disclosure. [Diagram 5] FIG. 1 is a flow diagram illustrating a process for authenticating for a VR system using multiple authentication factors according to one implementation of the present disclosure. [Figure 6] FIG. 1 is a block diagram illustrating a system for providing authentication for a VR system using multiple authentication factors, according to one implementation of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] As mentioned above, current virtual reality (VR) systems can use complex and cumbersome user interfaces.

[0015] Some implementations of the present disclosure provide multi-factor authentication suitable for VR systems using head mounted displays (HMDs). For example, the multi-factor authentication uses at least two of the following: (1) something the user knows, such as a password; (2) something the user has, such as a mobile phone or physical token; and (3) something the user is, such as the user's fingerprint, voice, or characteristic movements. In one implementation, the fingerprint can include a biometric signature. After reading these descriptions, it will be clear how to implement the present disclosure in various implementations and applications. Although various implementations of the present disclosure are described herein, it should be understood that these implementations are presented by way of example only and not limitation. Thus, the detailed description of the various implementations should not be interpreted as limiting the scope or breadth of the present disclosure.

[0016] In one implementation, multi-factor authentication involves entering a password and collecting a biometric fingerprint while wearing an HMD. For example, a user enters a password by gazing at a sequence of keys and selecting a sequence of data such as numbers, letters, symbols and / or images on a virtual keyboard displayed within the HMD. The biometric fingerprint can be collected by monitoring system movements, such as HMD movements (representing the user's head movements), or user movements, such as hand or body movements (captured by a camera).

[0017] In a specific implementation of first factor authentication (i.e., entering a password), the following steps may occur after a user puts on the HMD: First, an application of the VR system displays a virtual keyboard in the HMD. For example, a numeric keypad may be displayed in the HMD for entering a personal identification number (PIN).

[0018] Figure 1 is a layout 100 of a virtual keyboard or keypad displayed on a sphere according to one implementation of the present disclosure. In this implementation, there is a virtual keypad (which can be configured as an ordered keyboard or a random display of numbers) at a fixed location within the sphere, on top of which the HMD displays a keypad for the user to see. In one implementation, a 6-digit PIN can be used as the password. Thus, the application displays a 10-button keypad in the hemisphere in front of the user. The order of the 10 buttons is randomized.

[0019] In one implementation, a user moves their head towards a button (or key or other displayed virtual item) to select a key on the keypad to enter a password. This results in the button gazed at by the user being highlighted as a select button. The user confirms the select button by making a predefined head gesture of approval, such as nodding their head. The highlighting of the select button can be deselected by a predefined head gesture of deselection, such as shaking their head. Thus, the user enters a password by successively gazing at the key to be selected and confirming / deselecting it using a head gesture.

[0020] In one implementation, for security reasons, the layout of the virtual keypad is randomized at least for each new password entry. This layout randomization thus prevents unauthorized persons from revealing the password (by watching head movements). In some implementations, the layout can also be randomized after each key selection to prevent information (i.e., the password) from being revealed. In another implementation, the password is not alphanumeric, but a series of pictograms that should be arranged horizontally.

[0021] In a particular implementation of second factor authentication (i.e., biometric authentication), an application in the VR system measures biometric information to generate a biometric fingerprint. As with walking or handwritten signatures, the user's head movements for known gestures are unique to that user. For example, an approval head gesture and a deselect head gesture can serve this purpose. In one implementation, for both approval and deselect gestures, the biometric information is measured as acceleration projected on two axes of the head. The parameters of these gestures (e.g., acceleration) when normalized form a unique biometric fingerprint.

[0022] In a biometric authentication system, an initial enrollment phase records multiple instances of gestures or movements. During the initial enrollment phase, the user repeatedly performs approval and deselection head gestures. The enrollment phase then normalizes these inputs and averages them to generate a reference fingerprint. The application records the biometric fingerprints captured consecutively during key selection.

[0023] In one implementation, an application of the VR system can access a fingerprint database. The fingerprint database holds a record for each registered user that includes a user ID (which uniquely identifies the user) and the user's reference fingerprint. In one implementation, the reference fingerprint is stored as a set of data AR j ={δx j , δy j}, where δx represents azimuth acceleration and δy represents elevation acceleration. This set of data is established after a typical biometric enrolment phase where a user performs multiple recognition head gestures and the enrolment performs normalisation and statistical calculations to extract an "average" fingerprint for that user.

[0024] To authenticate a user, the application can verify two conditions: (1) the password entered via the virtual keypad matches the user's password, and (2) at least one of the measured biometric fingerprints matches the user's reference fingerprint (i.e., the user is verified). In one implementation, a fingerprint match means that the similarity between the measured biometric fingerprint and the reference fingerprint is within a specified limit.

[0025] 2 is a layout 200 of a virtual keypad displayed on a sphere but projected in polar coordinate form, according to one implementation of the present disclosure. The horizontal axis (i.e., x-axis) represents the azimuth angle, and the vertical axis (i.e., y-axis) represents the elevation angle. The origin is the orientation of the HMD at the start of the authentication process.

[0026] In one implementation, each displayed button or key (i.e., K i ) is K i ={x i , y i , V i}, where x i represents the azimuth angle in degrees of the center of the displayed key, and x i ∈]-90,90[, and y i represents the elevation angle of the center of the displayed key in degrees, and y i ∈]-90,90[, and V i represents the key value, Vi∈{0..9}. Therefore, in the implementation shown in Figure 2, K 1 = {0,30,0}, and K 4 ={-60,0,6}.

[0027] In one implementation, the HMD measures the user's head orientation (i.e., gaze) in polar coordinates {x user , y user} to the application periodically. Thus, initially, the application determines if the user is K if both conditions in the following equation [1] are true: i It is determined that you are looking at TIFF0007661419000001.tif10150[1] Here, R represents a fixed value. In one implementation, the value R is the radius of a circle that surrounds the key.

[0028] In one implementation, the process of selecting a key is described as follows:

[0029] Initially, when the application determines that the user is gazing or fixating on a key (i.e., formula [1] is satisfied for one value of i), it graphically highlights the key that is being gazed upon. When the user moves their eyes away from the key (i.e., formula [1] is no longer satisfied for the corresponding value of i), the application removes the highlighting.

[0030] Then, if the key is highlighted for a given period of time (e.g., 100 mS), the application enters a head gesture detection mode, which has three submodes. In the first submode, if an approval head gesture is detected, the application detects the value V i The application records the key selection using a key selection recorder. The user can be informed of the key selection by visual and audible signals. The application also records the measured biometric fingerprint. In the second submode, if a deselect head gesture is detected, the application removes the highlighting of the selected key. In the third submode, if no known gesture is detected, the application removes the current highlighting.

[0031] In one implementation, the password retrieval process waits for the user to select a predetermined number of keys (e.g., until six keys have been selected), after which the application enters the authentication verification phase.

[0032] In another implementation, the HMD is configured to include an eye gaze or eye movement tracking system that registers the direction of gaze or eye movement within the visor rather than the position of the HMD. user , y user} is a combination of the HMD position and gaze direction. Thus, this implementation allows key selection with reduced head movement. This implementation may be particularly useful for relatively large keyboards (e.g., larger than a numeric keypad with 10 keys). In this implementation, only the acceleration of the HMD is used to derive the measured biometric fingerprint (i.e., eye tracking information is not used to derive the measured biometric fingerprint).

[0033] In one implementation, in detecting head gestures, a nod of the user's head is considered an approval head gesture and a side-to-side shake is considered a deselection head gesture.

[0034] In one implementation, an application detects head gestures by recording head acceleration in a polar coordinate system every n milliseconds for a given number of seconds. Thus, the application records a series of samples A j ={δx j , δy j}, where δx j represents the azimuthal acceleration in the interval j, and δy j represents the elevation angular acceleration in the same interval j.

[0035] In one implementation, the process of detecting the accept and deselect head gestures can be divided into four phases. For example, phase 1 represents a pause where the HMD is "stationary" (the user is not perceptibly moving). Phase 2 represents movement in a first direction (e.g., up or down for accept head gestures and right or left for deselect head gestures). Phase 3 represents movement in a second direction opposite to the first direction (i.e., if the first direction is up then down is the second direction, if the first direction is right then left is the second direction, and vice versa). Phase 4 represents a pause where the HMD is "stationary".

[0036] In one implementation, the above-mentioned phases are characterized by a set of parameters (listed below). ε still = Pause acceleration threshold. ε move = Shake acceleration threshold. Δ still = minimum duration of a pause (i.e., phases 1 and 4). This parameter (Δ still ) has units of measurement periods. Thus, the actual minimum duration of a pause is Δ still ×n milliseconds. Δ mov1 = minimum duration of the first movement (i.e., phase 2). Δ mov2 = minimum duration of the second movement (i.e., phase 3). Δ max = maximum duration of each phase (i.e., the phase is Δ max If it does not complete within ×n milliseconds, the phase fails.

[0037] Sample A j belongs to one of the following categories: stationary, up, down, right, left or bad. The following set of equations defines this categorization: TIFF0007661419000002.tif39150[2]

[0038] Once the parameters have been defined as described above, successful completion of a phase or failure to complete a phase is defined by the following process.

[0039] 3 is a flow diagram illustrating a process 300 for detecting a phase according to one implementation of the present disclosure. In one implementation, the process 300 detects, at block 302, a Δ mov In phase 1 and phase 4, we verify whether there are consecutive samples with Δ mov = Δ still In phase 2, the category is down for the approval head gesture and right for the deselection head gesture, and Δ mov = Δ mov1In phase 3, the category is up for the approval head gesture and left for the deselection head gesture, and Δ mov = Δ mov2 If there are not enough consecutive samples, the detection fails at block 316.

[0040] If there are enough samples, the process waits until a sample has an unexpected category in block 324 or until the total number of analyzed samples reaches a threshold Δmax in block 330. In the first case (i.e., there is an unexpected category where the user starts another "movement", such as from up to down), the detection is successful in block 326. In the second case (i.e., the user stays in the same category for a long time, such as a long pause), the detection is successful but for too long in block 332.

[0041] In a particular implementation, first, in block 310, counter 1 and counter 2 are both set to 0. In block 312, a sample is taken, and in block 314, it is determined whether the sample is of the correct category. If the sample is not of the correct category, detection fails in block 316. On the other hand, if the sample is of the correct category, in block 318, counter 1 and counter 2 are both incremented. In block 320, counter 1 is incremented by Δ mov It is determined whether Counter 1 is less than Δ mov If it is less than Δ, then a further sample is taken in block 312. Otherwise, a sample is taken in block 322 and a determination is made in block 324 as to whether the sample taken is in the unexpected category or if the total number of samples analyzed is less than Δ maxIf, in block 324, it is determined that the obtained sample is of an unexpected category, then successful detection is indicated in block 326. On the other hand, if, in block 324, it is determined that the obtained sample is of the correct category, then counter 2 is incremented in block 328. Then, in block 330, it is determined whether the total number of analyzed samples has reached a threshold (i.e., counter 2<Δ max If the total number of samples reaches the threshold, then in block 332 it is indicated that the detection was successful but too long. Otherwise, in block 322 another sample is taken.

[0042] 4 is a flow diagram illustrating a process 400 for detecting an approval head gesture according to one implementation of the present disclosure. The process 400 describes a series of phase detection steps.

[0043] In the implementation shown in FIG. 4, the process 400 determines the duration of the movement (Δ mov ) is the minimum duration of the pause (Δ still ), in which case the expected category is "stationary" (phase 1). Phase 1 detection is then performed in block 412. If phase detection is unsuccessful in block 420, then a failed phase detection is indicated in block 422, i.e., the user's head has moved. If phase detection is successful but it is determined in block 424 that the pause duration was too long, then detection continues in block 412 until the correct phase 1 is determined in block 424.

[0044] In the implementation shown in FIG. 4, when phase 1 is detected, the process 400 determines the duration of the movement (Δ mov ) is the minimum duration of the first movement (Δ mov1), in which case the expected category is "down" (phase 2). Phase 2 detection is then performed in block 432. If phase detection fails in block 434, then the overall detection failure is indicated in block 436 and the key is deselected. If phase detection is successful but the duration of the first movement (Δ mov1 ) was determined to be too long, a total failure of the detection is indicated in block 456 and the key is deselected.

[0045] In the implementation shown in FIG. 4, when phase 2 is detected, the process 400 determines the duration of the movement (Δ mov ) is the minimum duration of the second movement (Δ mov2 ), in which case the expected category is "up" (phase 3). Phase 3 detection is then performed in block 442. If phase detection is unsuccessful in block 444, then the overall detection failure is indicated and the key is deselected in block 446. If phase detection was successful but the duration of the second movement (Δ mov2 ) was determined to be too long, a total failure of the detection is indicated in block 456 and the key is deselected.

[0046] In the implementation shown in FIG. 4, when phase 3 is detected, the process 400 determines the duration of the movement (Δ mov ) is the minimum duration of the pause (Δ still), in which case the expected category is "stationary" (phase 4). Phase 4 detection is then performed in block 452. If phase detection fails in block 454, overall detection failure is indicated and the key is deselected in block 456. If phase detection is determined to be successful regardless of its length in block 454, overall detection success is indicated in block 458.

[0047] 4 can be extended to a second process for detecting deselection head gestures, where phases 2 and 3 thus use head shake parameters instead of head nod parameters of process 400.

[0048] In an alternative implementation to that shown in FIG. 4, the nodding direction is selected to be up in phase 2 and the opposite (ie, down) in phase 3.

[0049] 5 is a flow diagram illustrating a process 500 for authenticating for a VR system using multiple authentication factors according to one implementation of the disclosure. In one implementation, the process 500 collects a password for a first one of the multiple authentication factors entered into a virtual keyboard by a user selecting a sequence of data including numbers and / or images on the virtual keyboard displayed on a display of the VR system at block 510. A biometric fingerprint for a second one of the multiple authentication factors is then collected from the movement of the VR system at block 520.

[0050] 6 is a block diagram illustrating a system 600 for performing authentication for a VR system using multiple authentication factors, according to one implementation of the present disclosure. In one implementation, the system 600 includes a first authentication collector element 610 and a second authentication collector element 620. In one implementation, the first element of the authentication collector 610 is configured to collect a password entered into a virtual keyboard by a user selecting a sequence of data including numbers and / or images on the virtual keyboard displayed on a display of the VR system. In one implementation, the second authentication collector element 620 is configured to collect a biometric fingerprint from the movement of the VR system.

[0051] The description of the implementations disclosed herein is provided to enable any person skilled in the art to make or use the invention. Numerous modifications of these implementations will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other implementations without departing from the spirit or scope of the invention. Thus, the present disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0052] Various implementations of the present disclosure are realized in electronic hardware, computer software, or a combination of these technologies. Some implementations include one or more computer programs executed by one or more computing devices. In general, a computing device includes one or more processors, one or more data storage components (e.g., volatile or non-volatile memory modules, and persistent optical and magnetic storage devices such as hard disk drives and floppy disk drives, CD-ROM drives, and magnetic tape drives), one or more input devices (e.g., game controllers, mice, and keyboards), and one or more output devices (e.g., display devices).

[0053] A computer program typically includes executable code that is stored on a computer-readable storage medium and copied into memory at run time. This code is executed by at least one processor reading the program instructions from memory in a predetermined order. When executing the program code, the computer receives data from input and / or storage devices, performs operations on the data, and provides resulting data to output and / or storage devices.

[0054] Those skilled in the art will appreciate that the various exemplary modules and method steps described herein can be implemented as electronic hardware, software, firmware, or a combination thereof. To clearly illustrate this compatibility between hardware and software, the various exemplary modules and method steps have been described herein generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention. Additionally, grouping of functionality within a module or step is for ease of description. Certain functionality may also be moved from one module or step to another without departing from the disclosure.

[0055] A particular implementation of the present disclosure does not necessarily require all features of each of the above-described embodiments. Moreover, it should be understood that the description and drawings presented herein are representative of the subject matter broadly contemplated by the present invention. Moreover, it should be understood that the scope of the present disclosure fully encompasses other implementations that would be apparent to one skilled in the art, and is therefore not to be limited except by the scope of the appended claims. [Explanation of symbols]

[0056] 300 processes 302 Block 310 Set counter 1=0, set counter 2=0 312 samples taken 314 Correct category? 316 Not Detected 318 Increment counter 1 and counter 2 320 Counter 1 <δmov?322 samples taken 324 Correct category? 326 Detection OK 328 Increase counter 2 330 Counter 2 <δmax? 332 Detection OK and long< / δmax?

Claims

1. 1. A method for providing authentication for a virtual reality (VR) system using multiple authentication factors, comprising: collecting a password for a first of the plurality of authentication factors, the password being entered into the virtual keyboard by a user gazing at and selecting a series of data including numbers, letters or images on a virtual keyboard randomly projected onto a polar coordinate system on a display of the VR system, each data item of the series of data projected onto the display being represented by an azimuth angle and an elevation angle; collecting a biometric fingerprint for a second of the plurality of authentication factors from a movement of the VR system, the biometric fingerprint being collected from detecting head gestures of the user including head movements with the VR system, the biometric fingerprint being measured as accelerations projected onto two axes of the polar coordinate system; Measuring a first measured acceleration of a predetermined approval head gesture; Measuring a second measured acceleration of the predetermined deselection head gesture; normalizing the first measured acceleration and the second measured acceleration to generate the projected acceleration; The method according to claim 1, further comprising:

2. The VR system is a head-mounted display (HMD). The method of claim 1.

3. displaying the virtual keyboard on the hemisphere at a fixed position within the hemisphere. The method of claim 1.

4. the virtual keyboard is arranged as a random display of at least 10 keys on the hemisphere; The method according to claim 3.

5. the password is entered by gazing in the direction of a button on the virtual keyboard and selecting the button; The method of claim 1.

6. the selected button is confirmed by performing the predetermined confirmation head gesture. The method according to claim 5.

7. The selection of the selected button is deselected by performing the predefined deselection head gesture. The method according to claim 6.

8. the biometric fingerprint is collected from the VR system's movements as reflected in the predefined approval head gesture and the predefined deselection head gesture; The method of claim 7.

9. The step of collecting a biometric fingerprint includes: measuring accelerations of the predefined approval head gesture and the predefined deselection head gesture; normalizing the acceleration; The method of claim 7, comprising:

10. comparing the collected biometric fingerprint with fingerprints in a fingerprint database; identifying the user when a fingerprint match occurs; The method of claim 1 further comprising:

11. A fingerprint match means that the similarity of the fingerprint match between the collected biometric fingerprint and a reference fingerprint is within a specified limit. The method of claim 10.

Images