Systems and methods for gesture-based authentication

Abstract

An authentication sever can receive sets of training sensor data instances from a plurality of sensors of an authentication system in response to a user initiating a plurality of training gestures. The sets of training sensor data instances associated with a set of authentication features. For each set of the sets of training sensor data instances, the authentication computer can determine distances between each pair of training sensor data instances in the set of training sensor data instances, and determine a similarity score for the set of training sensor data instances based on the distances. The authentication computer can determine a subset of the set of authentication features based on analyses of the similarity scores. The authentication computer can create an authentication model based on the subset of the set authentication features and use the model in determining whether a user can be authenticated.

Description

SYSTEMS AND METHODS FOR GESTURE-BASED AUTHENTICATIONCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 585,457, filed September 26, 2023, which is herein incorporated by reference in its entirety for all purposes.BACKGROUND

[0002] Users can interact with a virtual environment via one or more sensors of an augmented or virtual reality system. For example, a user can manipulate one or more hand-held controllers, a headset, a mobile device, and the like to perform operations in a virtual environment or to interact with components of a virtual environment. In one example, users can interact with resource providers in the virtual environment to obtain or gain access to resources. For example, a user can interact with a resource provider or other entity via based on sensor data associated with one or more sensors of a virtual reality system. The user can be authenticated and granted access to a secure resource based on, for example, a gesture performed by the user and recorded by the one or more sensors of the virtual reality system. Once access is granted, the user can interact with a remote system, e.g., a resource provider system, via the secure application.

[0003] Virtual environments present unique opportunities for user authentication because of the number of sensors from which unique data instances can be collected. For example, rather than using a password or PIN to authenticate the user, the user can perform a gesture with one or more controllers of a virtual or augmented reality system. However, there are challenges to using sensor data to authenticate a user because of the amount and complexity of data that can be collected by a virtual or augmented reality system. For example, data from multiple sensors can be collected. However, the collection of data from every possible available sensor may be unnecessary and may also be computationally expensive. Also, users have different characteristics. Thus, one set of features that may beoptimal to authenticate one user may not be optimal for another person. As an illustration, one person’s arm movement may be the best authentication feature for them. Whereas, another person’s eye movement may be the best authentication feature for that person.

[0004] Embodiments of the present application address these and other problems individually and collectively.SUMMARY

[0005] One embodiment of the invention includes a method. The method includes: in response to a user initiating a plurality of training gestures, receiving, by a computer, sets of training sensor data instances from a plurality of sensors in an authentication system, the sets of training sensor data instances associated with a set of authentication features; for each set of the sets of training sensor data instances: determining, by the computer, distances between each pair of training sensor data instances in the set of training sensor data instances, and determining, by the computer, a similarity score for the set of training sensor data instances based on the distances; determining, by the computer, a subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances; and creating, by the computer, an authentication model based on the subset of the set authentication features.

[0006] Another embodiment of the invention includes an authentication computer comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform a method comprising: in response to a user initiating a plurality of training gestures, receiving, from an authentication system, sets of training sensor data instances from a plurality of sensors of the authentication system, the sets of training sensor data instances associated with a set of authentication features; for each set of the sets of training sensor data instances: determining distances between each pair of training sensor data instances in the set of training sensor data instances, and determining a similarity score for the set of training sensor data instances based on the distances; determining a subset of the set of authentication features based onanalyses of the similarity scores associated with the sets of sensor training data instances; and creating an authentication model based on the subset of the set authentication features.

[0007] Another embodiment of the invention includes an authentication system comprising: a processor; a plurality of sensors; a memory device; and a computer- readable medium coupled to the processor, the computer-readable medium comprising code executable by the processor for implementing a method comprising: in response to a user initiating a gesture, collecting sets of test sensor data instances from the plurality of sensors, the sets of training sensor data instances associated with a set of authentication features; transmitting, to an authentication computer, an authentication request message comprising the sets of test sensor data; receiving, from the authentication computer, an authentication response message indicating whether the user has been authenticated based on application of an authentication model to the sets of test sensor data.

[0008] Further details regarding embodiments of the invention can be found in the Detailed Description and the Figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 shows a block diagram of a system according to an embodiment of the invention.

[0010] FIG. 2 shows a block diagram of an exemplary authentication computer according to an embodiment of the invention.

[0011] FIG. 3 shows a flow diagram illustrating a process according to an embodiment of the invention.

[0012] FIG. 4 shows a flow diagram illustrating an enrollment process according to an embodiment of the invention.

[0013] FIG. 5 shows a flow diagram illustrating an authentication process according to an embodiment of the invention.

[0014] FIGs. 6A-6D illustrate example screenshots of an authentication process according to an embodiment of the invention.DETAILED DESCRIPTION

[0015] Embodiments include systems and methods for authenticating a user based on a gesture performed in a virtual environment. For example, a user may swipe, tap, or dip a virtual payment card, or draw their signature in a virtual environment to make an in-game / in-app payment. Embodiments can use built-in sensors in equipment, such as a VR headset and a VR controller, to capture movements (sliding up, down, right, left, tapping, pushing, pulling) performed by the user. From the collected data, an authentication system can extract features (e.g., handheld device gripping pressure, handheld device key pressure, key hold time, and time between interaction elements) that could uniquely identify a user and be used to authenticate the user.

[0016] In some embodiments, the user can initiate an enrollment phase, during which, sensors of the authentication system collect and record data associated with the user. For example, the user can press a trigger of a handheld device to begin the gesture recording and can press the trigger again to indicate the gesture is complete. During an enrollment phase the user can be asked to repeat the gesture multiple times. The authentication system can transmit the gesture data to an authentication computer for analysis. For example, the authentication computer can select stable features for the user, which are stored as a gesture template.

[0017] When the user performs the gesture at a subsequent time, the collected sensor data can be transmitted to the authentication computer. The authentication computer can extract the features and feed the features to a distance function. Authentication is successful if a majority of the collected features have a distance less than a threshold to the stored features in the gesture template.

[0018] Prior to discussing specific embodiments of the invention, some terms may be described in detail.

[0019] A “user” may include an individual. In some examples, a user may be associated with one or more personal accounts, authentication systems, and / or mobile devices. The user may also be referred to as a cardholder, account holder, or consumer in some embodiments.

[0020] An “authentication system” may be a system of one or more components that a user can interact with. An authentication system can be a VR system or AR system. Components of an authentication system can include a handheld controller, a headset, a mobile device, a gaming console, a motion capture device, and the like. In some embodiments, the authentication system and / or a component of the authentication system may include a display, a memory, a processor, a computer- readable medium, and any other suitable component.

[0021] A “sensor” may include any suitable electronic device for collecting data. Examples of a sensor include: an accelerometer, a gyroscope, a magnetometer, an infrared sensing device, a GPS locator, a light sensor, a camera, a microphone, a button or trigger, a trackpad or mouse, a touchscreen, a keypad or keyboard, and the like. The components of an authentication system can include one or more sensors for collecting data.

[0022] An “authentication request message” may be an electronic message that communicates a request to authenticate a user to an authentication computer. In some examples, an authentication request message is sent to the authentication computer to cause the authentication computer to authenticate a user based on data collected by one or more sensors of an authentication system. The authentication request message may include encrypted sensor data, raw sensor data, and / or user information (e.g., username, account number, authentication system ID information, etc.).

[0023] An “authentication response message” may be an electronic message that communicates whether or not a user has been authenticated by the authenticaiton server based on the data of the authentication request message. In some examples, an authentication response message is sent to the secure application to cause the secure application to grant or deny access to the user via the authentication system.

[0024] The term “authentication” generally refers to a process of establishing confidence in the identity of a user or a computer. Certain electronic authentication systems may require a user to input authentication information (e.g., a password, a personal identification number (PIN), or other similar information) for authentication of the user. In other examples, one or more sensors of the authentication system can record data associated with the user performing a gesture or movement unique to the user. The authentication computer may authenticate the user by comparing the recorded data with stored data, e.g., gesture data received by the authentication computer during enrollment of the user.

[0025] “Authentication information” may include any data or information suitable to authenticate a user or device. Examples of authentication information may include sensor data associated with the performance of a gesture by the user, sensor data associated with a physical environment of the user, a password or passphrase, a secret key (e.g., a private key), a digital signature, biometric information, an indication that the device is storing certain information (e.g., biometric information), etc.

[0026] The term “gesture” may include a movement. For example, a gesture can be a movement performed by a user. In some examples, the gesture can be recorded via the sensors or one or more components of the authentication system.

[0027] A “training sensor data instance” may include sensor data. A training sensor data instance can include sensor data that is generated as a user performs a gesture. A training sensor data instance can include raw sensor data that is generated as the user performs the gesture using an authentication system. Sets of training sensor data instances can be associated with authentication features (e.g., trajectory, velocity, acceleration, etc.).

[0028] A “similarity score” may include a value that indicates likeness of two things. A similarity score can indicate a likeness of two training sensor data instances. A similarity score can be determined based on a distance between two training sensor data instances or can be based on an application of a similarity function to the determined distance.

[0029] “Authentication features” may include attributes or aspects of gestures used for authentication. Authentication features can include features determined from training sensor data instances and / or raw sensor data. Example authentication features include raw acceleration data from the sensors, raw location data (x,y,z) from the sensors, handheld gripping pressure, handheld key pressure, key hold time (VR and physical), inter-time, distance between multiple pressure points, gesture speed (in certain use cases can also be in VR), location of the payment object in the VR space (x,y,z of gesture with respect to the terminal), acceleration (within a defined interval), trajectory, hovering time, angle of holding the handheld and orientation throughout the movement, distance between the user’s two hands, distance between hand and head, distance from head to distance, and other data that can indicate distances, relationships, pressures, movements, etc. in a three- dimensional space (in virtual reality space and / or real space).

[0030] An “authentication model” may include a mathematical description of authentication. An authentication model can be associated with a particular user. An authentication model can be used to authenticate a user in response to a user attempting to launch or access secure functionality or resources of the secure application. An authentication model can be created using authentication features determined from training sensor data instances.

[0031] An “entropy value” may include a measure of a uniformness of a value. In information theory, an entropy of a random variable is the average level of information, surprise, or uncertainty inherent to the variable's possible outcomes. A low entropy value can correspond to a variable that has high certainty, and can then be more easily guessed or predicted. A high entropy value can correspond to a variable that has a low certainty, and can then be difficult to guess or predict.

[0032] An “authentication computer” may include a powerful computer or cluster of computers, and may be a server computer. The authentication computer can be a server, which services the requests of a client such as an authentication system. The authentication computer can be associated with a payment processor or other financial entity. For example, the authentication computer can be a large mainframe, a minicomputer cluster, or a group of servers functioning as a unit. In one example,the authentication computer may be a database server coupled to a Web server. The authentication computer may be coupled to a database and may include any hardware, software, other logic, or combination of the preceding for servicing the requests from one or more client computers.

[0033] A “processor” may include a device that processes something. In some embodiments, a processor can include any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to accomplish a desired function. The processor may include a CPU comprising at least one high-speed data processor adequate to execute program components for executing user and / or system -generated requests. The CPU may be a microprocessor such as AMD's Athlon, Duron and / or Opteron; IBM and / or Motorola's PowerPC; IBM's and Sony's Cell processor; Intel's Celeron, Itanium, Pentium, Xeon, and / or XScale; and / or the like processor(s).

[0034] A “memory” may be any suitable device or devices that can store electronic data. A suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories may comprise one or more memory chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation.

[0035] FIG. 1 shows a system 100 comprising a number of components. The system 100 comprises an authentication system 110 associated with and operated by a user. The system 100 further comprises an authentication computer 115, which may be embodied by one or more computers.

[0036] In some examples, the authentication system 110 can include the authentication computer 115. For example, the authentication computer can be housed in the authentication system 110 such that the user’s identification information and the generated sensor data do not have to be transmitted via a network between the authentication system 110 and the authentication computer 115

[0037] The authentication system 110 may be capable of interacting with the authentication computer 115. Also, the authentication system 110 and the authentication computer 115 may be in operative communication with each other through any suitable communication channel or communications network. Suitable communications networks may be any one and / or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as, but not limited to a Wireless Application Protocol (WAP), l-mode, and / or the like); and / or the like.

[0038] Messages between the computers, networks, servers, and devices may be transmitted using a secure communications protocols such as, but not limited to, File Transfer Protocol (FTP); HyperText Transfer Protocol (HTTP); Secure Hypertext Transfer Protocol (HTTPS), Secure Socket Layer (SSL), ISO (e.g., ISO 8583) and / or the like.

[0039] An example of the authentication system 110 is a virtual reality system or an augmented reality system configured to enable the user to interact with a virtual environment and capable of executing one or more applications stored thereon. For example, the authentication system 110 may be configured to execute an enrollment application 120 and a secure application 125. The enrollment application 120 may be an application associated with the authentication computer 115. The enrollment application 120 can be used to collect and transmit sensor data to the authentication computer 115. The secure application 125 may be an application associated with a resource provider, such as a banking or e-commerce application through which the user can complete transactions.

[0040] The authentication system 110 may include a processor 110A operatively coupled to a computer readable medium 110B (e.g., one or more memory chips, etc.), sensors 110C such as buttons, gyroscopes, accelerometers or the like, an output device 110D (e.g., a display, a speaker, etc.) and a network interface 110E.

[0041] The computer readable medium 11 OB may include instructions or code, executable by a processor (e.g., the processor 110A). The instructions can include instructions for recording sensor data and communication with the authentication computer 115 to enroll the user in an authentication program using a gesture performed by the user. The instructions may also include instructions for communicating with an authentication computer 115 to authenticate the user to enable the user to access a secure application 125 of the authentication system 110.

[0042] For example, the computer readable medium 110B can include a series of instructions that, when executed, cause the processor 110A to communicate with the authentication computer 115 to enroll and / or authenticate the user via the authentication system 110. The computer readable medium 110B of the authentication system 110 can include the enrollment application 120 and the secure application 125. The authentication system 110 can execute the enrollment application 120 to record sensor data (e.g., from the sensors 110C) associated with performance of a gesture by the user. The authentication system 110 can transmit the recorded sensor data to the authentication computer 115 to be analyzed and stored by the authentication computer 115 for future use in authenticating the user.

[0043] In response to the user initiating a plurality of gestures, the secure application 125 can generate an authentication request message for transmittal to the authentication computer 115. The authentication request message can include raw sensor data or encrypted sensor data collected by the sensors 110C as the user performs the gestures. The authentication computer 115 can use the sensor data to authenticate the user based on the sensor data matching, within a threshold, previously stored sensor data (e.g., the sensor data received and stored by the authentication computer 115 during user enrollment).

[0044] The computer readable medium 110B can also include instructions stored thereon that, when executed by the processor 110A, cause the authentication system 110 to perform a method including: in response to a user initiating a gesture, collecting sets of test sensor data instances from the plurality of sensors, the sets of training sensor data instances associated with a set of authentication features; transmitting, to an authentication computer, an authentication request messagecomprising the sets of test sensor data; receiving, from the authentication server, an authentication response message indicating whether the user has been authenticated based on application of an authentication model to the sets of test sensor data.

[0045] Returning to FIG. 1, the user may use the authentication system 110 to enroll in an authentication program of the secure application 125. For example, the user can initiate the enrollment application 120 to record a plurality of gestures that can be used in authenticating the user with the secure application 125. In some embodiments, the enrollment application 120 is included in the secure application 125

[0046] The user can initiate the enrollment application 120, for example, by navigating to the secure application 125 or to the enrollment application 120 in the virtual environment. The enrollment application 120 can instruct the user, via an output device 110D of the authentication system (e.g., a display or speaker), to begin performing a gesture. The gesture can be, for example, a signature or figure drawn with a handheld controller of the authentication system 110. The enrollment application 120 can instruct, via the authentication system 110, the user to repeat the gesture a predetermined number of times. In another embodiment, the enrollment application can instruct the user to repeat the gesture until a threshold amount of sensor data is recorded.

[0047] The enrollment application 120 can generate an enrollment message containing the recorded sensor data and transmit the enrollment message to the authentication computer 115. The authentication computer 115 can analyze the received sensor data to generate and store a signature associated with the user. The signature can be used to authenticate the user in future attempts to log in to the secure application 125.

[0048] The user may be able to use the authentication system 110 to interact with a secure resource in a virtual environment via the secure application 125. For example, the secure application 125 can enable the user to conduct transactions, manage an account, or execute other operations in a virtual environment. In a virtualenvironment, the user can initiate an interaction with the secure application 125 by, for example, approaching or otherwise navigating to a virtual kiosk, icon, or storefront in the virtual environment. In response, the secure application 125 can prompt the user to provide authentication information, e.g., by performing a gesture or a plurality of gestures. In one example, in response to the user opening or otherwise launching the secure application 125, during a login process, the sensors 110C can collect data as the user performs the plurality of gestures. In some embodiments, the user can initiate the sensors 110C to record the data associated with the gestures. In another embodiment, the sensors 110C can record data during a predetermined time interval during the login process.

[0049] Once gesture recording is complete, the secure application 125 can generate an authentication request message including the recorded sensor data. In some embodiments the authentication request message can include identity information associated with the user such as, for example, a name, an account number, an authentication system model or serial number, an IP address, and the like.

[0050] The authentication computer 115 can receive the authentication request message and use the identity information to query a database of user signatures. The authentication computer 115 can use the received sensor data to generate a signature and compare the generated signature with a stored signature to determine whether or not the user is enrolled with the secure application 125. The authentication computer 115 can then generate an authentication response message to transmit to the secure application 125. The authentication response message can indicate whether the signature generated from the received data matched, within a threshold, the stored signature.

[0051] If the generated signature does not match the stored signature, the authentication computer 115 can generate an authentication response message communicating to the secure application 125 to deny the user access to the secure application 125. In some embodiments, if the generated signature does not match the stored signature, the authentication computer 115 can generate an authentication response message causing the secure application 125 to request theuser repeat the gesture one or more times. Data associated with the repeated gesture can be transmitted to the authentication computer 115 for analysis, and the user may be denied access if the generated signature does not match the stored signature after a certain number of attempts.

[0052] If the generated signature matches the stored signature, the authentication computer 115 can generate an authentication response message communicating to the secure application 125 to grant the user access to the secure application 125. Granting access to the secure application 125 can refer to enabling access to secure resources or functionality via the secure application 125. In another embodiment, granting access can include enabling the user to complete a transaction via the secure application 125.

[0053] The authentication computer 115 may include data processing subsystems, networks, and operations used to support and deliver authentication services. For example, the authentication computer 115 may include a server coupled to a network interface (e.g., by an external communication interface), and databases of information. The authentication computer 115 can be managed, for example, by a transaction processing network, a financial institution, a resource provider, an authentication service, or another entity. The authentication computer 115 may use any suitable wired or wireless network, including the Internet.

[0054] An example of the authentication computer 115 is illustrated in FIG. 2. The authentication computer 115 may include a processor 115A operatively coupled to a computer readable medium 115B (e.g., one or more memory chips, etc.), memory 115C, a network interface 115D, and a data repository 115G.

[0055] The computer readable medium 115B may include instructions or code, executable by a processor (e.g., the processor 115A). The instructions may include instructions for communicating with the authentication system 110 to authenticate the user via the authentication system 110, and instructions for any other suitable function as described herein. For example, the computer readable medium 115B may store an authentication model 115E that is generated based on the sensor data received from the authentication system 110. The authentication model 115E can beassociated with a particular user and can be used to authenticate the user in response to a user attempting to launch or access secure functionality or resources of the secure application 125.

[0056] In an example, the authentication computer 115 can receive sets of training sensor data instances collected by a plurality of sensors (e.g., sensors 110C of the authentication system 110) as part of an enrollment process. The sets of training sensor data instances can be associated with a set of authentication features. The authentication computer 115 can analyze each of the sets of training sensor data instances by determining distances between each pair of training sensor data instances in the set of training sensor data instances, and determining a similarity score for the set of training sensor data instances based on the distances. Based on this analysis, the authentication computer 115 can determine a subset of authentication features which provide the basis for the authentication model 115E. For example, the subset of authentication features can include features having the greatest mean similarity scores and / or the lowest variance of similarity scores.

[0057] The authentication computer 115 can include a data repository 115G, which can be a physical database, a cloud database, or a data warehouse. The data repository can store the authentication model associated with each enrolled user as a signature in signature data 1151. In some embodiments, the data repository 115G can store the received training sensor data instances as training data 115H.

[0058] In some embodiments, the training data 115H associated with each user can be supplemented with newly received test sensor data instances when the user is successfully authenticated. This data can be used to periodically refresh the authentication model associated with the user, which is then updated and stored as a signature in signature data 1151. By refreshing the authentication model 115E, variable changes in the authentication features can be captured and accounted for. For example, refreshing the authentication model can ensure that the user can authenticate by performing the enrolled gesture as the user’s pattern of movement changes over time, e.g., due to age or mobility issues.

[0059] When the authentication computer 115 receives an authentication request message from the authentication system 110, the authentication computer 115 can use the authentication model 115E to process sets of test sensor data instances received in the authentication request message to determine whether to authenticate the user. For example, a user can be authenticated by the authentication computer 115 if the sets of test sensor data instances fall within a pre-defined authentication threshold.

[0060] The computer readable medium 115B can also include an application programming interface (API) 115F allowing for secure connection and communication with the authentication system 110. The API 115F can define the interaction between the authentication system 110, via the enrollment application 120 or the secure application 125, and the processor associated with the authentication computer 115. The API 115F can enhance security of data communicated between authentication system 110 and the authentication computer 115

[0061] The memory 115C can be a database, or other memory, physical storage device, or cloud-based storage device. In some embodiments, the memory 115C can include the data repository 115G and can store identifying information and signature data associated with the user. The signature data can be used in addition to or in lieu of a username / password combination, biometric information, or other user-specific authentication information. Accordingly, the user can be authenticated by the authentication computer 115 through manipulation of one or more components (e.g., a handheld controller) of the authentication system 110.

[0062] The computer readable medium 115B can also include instructions stored thereon that, when executed by the processor 115A, cause the authentication computer 115 to perform a method including: in response to a user initiating a plurality of training gestures, receiving, from an authentication system, sets of training sensor data instances from a plurality of sensors of the authentication system, the sets of training sensor data instances associated with a set of authentication features; for each set of the sets of training sensor data instances determining distances between each pair of training sensor data instances in the setof training sensor data instances, and determining a similarity score for the set of training sensor data instances based on the distances; determining a subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances; and creating an authentication model based on the subset of the set authentication features.

[0063] A process 300 according to examples of the present application is described with respect to FIG. 3.

[0064] The process 300 provides authentication of a user, via a gesture performed by the user using the authentication system 110. In a specific example, the process 300 may be used to authenticate the user in response to an attempt to access secure resources or functionality of the secure application 125.

[0065] At step S302, the user can initiate a plurality of training gestures using the authentication system 110. For example, the user can approach or otherwise navigate to a virtual kiosk, icon, or virtual storefront in a virtual environment to access the secure application 125. The user can launch the enrollment application 120 by logging in to the secure application 125 for the first time or by selecting functionality of the secure application 125 to enroll in a gesture-based authentication program.

[0066] The gesture can be any movement in three-dimensional space and can be performed using any component of the authentication system 110. For example, the gesture could be the user’s signature, a drawing of a shape indicated by the enrollment application 120, inputting a PIN on a virtual keypad, manipulating a virtual card (e.g., swiping, dipping, or tapping a virtual card at a virtual kiosk or a virtual point of sale terminal), and the like. As the user performs the gesture, sensors 110C of components of the authentication system 110 can record data associated with the gesture.

[0067] As an example, the sensors 110C can collect and record sets of training sensor data instances, which are generated as the user executes the gesture. In one example, a handheld controller of the authentication system 110 can convert physical movements into digital movements and can include a gyroscope and anaccelerometer, which can record data such as trajectory and velocity associated with execution of the gesture. In another example, a headset of the authentication system 110 can include an accelerometer, a gyroscope, a magnetometer, and eye tracking sensors to capture location, motion, orientation, and direction needed to construct the authentication model 115E associated with the gesture. In another example, the authentication system can collect data associated with the user or the user’s physical environment, such as ambient light levels, GPS coordinates, height of headset from the ground, distance from the headset to a handheld controller, a distance between two or more handheld controllers, and the like.

[0068] In one example, the user can approach a virtual kiosk in a virtual environment by using controls and / or sensors of the authentication system 110. The user can initiate enrollment in a gesture-based authentication program, which can cause the authentication system 110 to prompt the user to execute a gesture. The sensors 110C can automatically record data for a preset period of time or until the user indicates to the authentication system 110 that the gesture is complete.

[0069] An exemplary enrollment process 400 is further illustrated in FIG. 4. At block S402, the authentication system 110 can instruct the user to perform the gesture N times, where N is an integer greater than zero. As the user is performing the gesture N times, the sensors 110C of the authentication system collect sets of training sensor data instances. The training sensor data instances can include, for example, raw sensor data generated as the user performs the gesture using the authentication system 110. In another example, the training sensor data instances can be a number of dimensions of time-series raw signals from different sensors of the authentication system 110. The sets of training sensor data instances can be associated with authentication features 404, such as trajectory, velocity, and acceleration. Other authentication features can be extracted from the raw sensor data through wavelet transformation, fast Fourier transform (FFT), or other signal processing methods.

[0070] When the N repetitions of the gesture are complete, the authentication system 110 can generate an enrollment message including the sets of training sensor data instances to be transmitted to the authentication computer 115.

[0071] Returning to FIG. 3, at step S304, the authentication computer 115 can receive the sets of training sensor data instances transmitted by the enrollment application 120. For example, a user can perform a gesture to sign their name in a virtual reality environment ten times. The multiple gestures may produce trajectory data, velocity data, and acceleration data. A set of training data instances can be ten instances of trajectory data associated with the ten signing gestures. Another set of training data instances can be ten instances of velocity data associated with the ten signing gestures. Yet another set of training data instances can be ten instances of acceleration data associated with the ten signing gestures. The authentication computer can generate additional authentication features by performing signal processing on the sets of training sensor data instances. In some embodiments, the authentication computer 115 can generate a number of time-series signals from the raw sensor data. The time-series signals can be used by the authentication computer 115 to characterize user behavior and perform feature selection to find a subset set of features for an individual user that can be used to generate the authentication model for the user.

[0072] At step S306, for each set of training sensor data instances (e.g., training sensor data instance sets associated with authentication features 404 of FIG. 4), the authentication computer 115 determines distances of each pair of training sensor data instances in the set of training sensor data instances. The distance between two training sensor data instances can be, for example, a difference between the values of each instance. For example, with respect to the above noted illustration, for the set of velocity training data instances for the ten signing gestures may be V1 , V2, ...., V10. Differences can be determined for each pair of instances in the set of instances (e.g. V1-V2, V1-V3, V1-V4, and so on).

[0073] At step S308, the authentication computer 115 can determine a similarity score for the set of training sensor data instances for a particular feature (e.g., velocity) based on the determined distances. The similarity score can be based on the distances between each pair of instances or can be based on the application of a similarity function to the determined distances. The similarity score can indicate, for example, a measure of how alike the distances of the pairs instances for a givenfeature are. For example, with respect to the above noted illustration, for the set of velocity training data instances for the ten signing gestures may be V1 , V2, V10.Differences can be determined for each pair of instances in the set of instances (e.g. V1-V2, V1-V3, V1-V4, and so on). Similarity scores can be based on the differences. For example, if the differences for all pairs of velocity instances are very low, then those differences can be used to calculate a similarity score for the velocity feature of “10” where 10 means high similarity and 1 means very low similarity. In another example, a similarity score for a feature can be based on a mean and standard deviation of the distances (e.g., as shown in step S406 of Fig. 4) of the pair of instances associated with that feature.

[0074] Steps S306 and S308 can be repeated for each set of training sensor data instances. Accordingly, for each authentication feature, the authentication computer can determine a set of similarity scores.

[0075] At step S310, the authentication computer 115 determines a subset of authentication features based on the similarity scores. To determine the subset, the authentication computer 115 can, for each authentication feature, determine a mean and standard deviation of the similarity scores associated with that feature.

[0076] As illustrated in step S408 of FIG. 4, the authentication computer 115 can select the subset of authentication features by selecting the top K authentication features having the highest mean similarity scores and the lowest standard deviation, where K is an integer greater than zero.

[0077] In other embodiments, for the set of authentication features, the authentication computer 115 can calculate their entropy values for the signal associated with each authentication feature. For example, the authentication computer 115 can determine the entropy values by computing the Shannon entropy. If a signal’s entropy value is low, then the signal does not contain enough information for use in authenticating the user. Thus, the authentication computer 115 can rank the authentication features by their entropy values and remove the bottom percentage of the authentication features from the set of authentication features.

[0078] In another embodiment, some authentication features have small or no variance since the authentication features were not involved in the user’s motion when performing the gesture. As an example, a feature indicating geographic position or a height of a headset from the ground may not change while the user is standing stationary and performing a gesture. Therefore, the authentication computer 115 can remove authentication features that have exactly (or statistically similar) the same mean and variations.

[0079] Returning to FIG. 3, at step S312, the authentication computer 115 can create and store an authentication model based on the subset of authentication features. For example, as illustrated in steps S410A and S410B of FIG. 4, the authentication computer can store the means and the standard deviations of the K selected authentication features and can store the K selected authentication features as a gesture template associated with the user. For example, the gesture template and the means and the standard deviations can be stored in the data repository 115G of the authentication computer 115 as signature data 1151. The authentication model (e.g., the stored gesture sample and the means and the standard deviations of the K features) can be used to authenticate the user at a subsequent login attempt. For example, the authentication computer can use 6o (six sigma) methods to determine whether the user can be authenticated based on the authentication model.

[0080] At step S314, the authentication computer 115 can receive an authentication request message from the authentication system 110. The authentication request message can be generated in response to a user attempting to access the secure application 125 and performing a gesture using the authentication system 110. The authentication request message can include identification information associated with the user and can contain sets of test sensor data instances. The sets of test sensor data instances can be generated when the user performs a gesture using one or more components of the authentication system 110 (e.g., a headset and / or handheld controller) to gain access to the secure application 125. In some embodiments, the sets of test sensor data instancesincluded in the authentication request message correspond to the subset of authentication features. E

[0081] In response to receiving the authentication request message, as shown at step S502 of FIG. 5, the authentication computer 115 can query a database (e.g., signature data 1151) based on the identification information to locate a gesture template (e.g., an authentication model) associated with the user. The gesture template can also include a listing of the subset of features used to generate the template.

[0082] At step S504, the authentication computer 115 can compute the distances between the stored features and the features determined from the sets of test sensor data instances. The authentication computer 115 can then, for each set of test sensor data instances, determine the mean of the determined distances.

[0083] At step S506A, the authentication computer 115 can determine the similarity scores for the subset of features (e.g., the K selected features) based on the mean distance between the received feature and template feature associated with each of the K authentication features in the subset of authentication features. The similarity scores can be determined using similar methods to those described with respect to S406 of FIG. 4. In some embodiments, the similarity scores can be determined using dynamic time wrapping techniques.

[0084] At step S506B, the authentication computer can determine the mean and standard deviation of each authentication feature of the subset of authentication features based on the sets of test sensor data instances.

[0085] Returning to FIG. 3, at step S316, the authentication computer 115 can determine whether or not the user is authenticated based on the authentication model. For example, as illustrated at S508 of FIG. 5, the user can be authenticated if a threshold number of the similarity scores calculated at S506A fall within a predetermined threshold (e.g., two standard deviations) of the mean similarity scores for the authentication feature. In another embodiment, the authentication computer 115 can determine whether the user is authenticated by applying the authentication model to the sets of test sensor data instances.

[0086] In one example, applying the authentication model to the sets of test sensor data instances can include determining where the test sensor data instances are located relative to a statistical distributions of the test sensor data instances, determining whether each test sensor data instance meets a threshold with respect to a statistical distribution in the statistical distributions; and determining if a predetermined number of test sensor data instances meet one or more thresholds with respect to the statistical distributions.

[0087] At step S318, the authentication computer 115 can generate an authentication response message containing the results of the authentication and transmit the authentication response message to the authentication system 110. If the authentication response message indicates the user was authenticated based on the sets of test sensor data instances associated with the gesture, then the secure application 125 may complete a transaction or grant access to the user to restricted resources or functionality. If the authentication response message indicates the user was not authenticated based on the sets of test sensor data instances associated with the gesture, then the secure application 125 may decline a transaction or deny access to the user to restricted resources or functionality. In some embodiments, if the user is not authenticated, the user may be given a number of attempts to perform the gesture.

[0088] Another example of selecting the subset of authentication features from which to generate the authentication model for a user is described below.

[0089] In a Metaverse™ scenario experiment (e.g., in a particular virtual reality world), users can be authenticated purely based on their movement. The movement had six degrees of freedom: up; down; left; right; back; and forward. In this example, the authentication system 110 collected around 150 dimensions of time series raw signals from different sensors of left-hand handles, right-hand handles, headset, and aviator (e.g., in a device such as Oculus™). The collected raw signals tracked the users’ movement of head, eyes, left hand, and right hand in regard to three- dimensional position, velocity, rotation, and acceleration.

[0090] In some embodiments, signal processing transformation techniques were applied to the 150 dimensions of raw signals to extract additional information from the raw signals. The signal processing transformation techniques can be FFT, Wavelet, or other signal processing techniques.

[0091] Thus, the raw signals and transformed signals yielded 500 different time series signals. The 500 time series signals were then used to characterize user behavior and to perform feature selection to find the optimal set of features to be used in authenticating the user based on the gesture.

[0092] For a user to create an account (e.g., via the enrollment application 120), the user created an authentication motion (e.g., a gesture) and repeated the gesture a predetermined number of times (e.g., five times), during which the components of the authentication system 110 (e.g., the left- and right-hand handles, headset, and aviator) collected data associated with the gesture. These five trials of a gesture were represented as {Ti , T2, T3, T4, Ts}.

[0093] For one trial of the gesture (e.g., T1 ), there were m raw time series signals and n transformed signals based on FFT, Wavelet and other common signal processing methods. In this example, T1 included a set of signals: Sraw = {sraw_ti_i, Sraw_t1_2, ... , Sraw_t1_m} and Strans={Strans_t1_1 , Strans_t1_2, ... , Strans _t1_n}.

[0094] The five trials of the user’s motion generated ten pairs: (T1, T2); (T1 , T3); (T1, T4); (TI , TS); (T2, T3); (T2, T4); (T2, TS); (T3, T4); (T3, TS); and (T4, Ts). For each pair of trials, a similarity score was calculated between the two trials based on the set of signals Sraw and Strans. Thus, for example, the similarity for (Ti , T2) was the set of Similarity(sraw_ti_i , Sraw_t2_i ), ... Similarity(st rans_t1_n, Strans_t2_n ). Using this method, there were (m+n) similarity values for the Similarity measurement between Ti and T2. The similarity value calculation was repeated for each of the ten pairs of trials, yielding (m+n) similarity values for each trial.

[0095] From the ten pairs’ m+n) similarity values, the authentication computer 115 calculated a mean similarity score and standard deviation of the similarity scores to get (m+n) means and standard deviation. In some embodiments, theauthentication computer also determined other metrics, such as the skewness and kurtosis of the similarity scores.

[0096] From the set of authentication features, the authentication computer 115 filtered out features that did not contain enough information by calculating their entropy values. For example, if a signal’s entropy value was low, then the signal did not contain enough information for authentication purpose (e.g., to be statistically significant). The authentication features were ranked by their entropy values and the authentication computer 115 removed a predetermined percentage of the authentication features having the lowest entropy signals.

[0097] From the remaining features, the authentication computer removed the authentication features having a small variance or no variance. For example, small to no variance can indicate that these features were not involved in the user’s motion while performing the gesture. Therefore, the authentication features that have exactly the same mean and variations were removed from the set of authentication features.

[0098] Finally, for the remaining raw and transformed authentication features, the authentication computer 115 ranked the features by their means from high to low, and by their variations from low to high. The authentication computer 115 selected a predetermined number of authentication features (e.g., the top K features) to generate the subset of authentication features. This subset of authentication features formed the basis of the authentication model that was used to authenticate the user.

[0099] Because only the current user’s sensor data is used in the authentication feature selection process, other users’ information can be protected. The system can therefore minimize the leakage of other users’ sensitive authentication information. Additionally, the authentication feature selection process is efficient, and can be executed in an edge device (e.g., the authentication system 110). Since the authentication process can happen in the edge device, sensitive data is not transferred back to a cloud or separate server, thereby preserving user’s authentication information.

[0100] FIGs. 6A-6D are illustrations of an exemplary virtual environment for implementing gesture-based authentication. The virtual environment shown in FIGs. 6A-6D can be displayed to a user via the authentication system 110, which can be, for example, a virtual or augmented reality system.

[0101] In FIG. 6A, the virtual environment can include a virtual kiosk 600 through which a user can access the secure application 125. The virtual kiosk 600 can display a message prompting the user to perform a gesture in order to be authenticated, or in this example, in order to complete a transaction.

[0102] At FIG. 6B, the virtual kiosk 600 can display a virtual medium (e.g., a virtual signature pad) through which the user can perform the gesture (e.g., the signature) using a stylus or handheld controller of the authentication system 110.

[0103] At FIG. 6C, the virtual kiosk 600 can display representation of the gesture being performed by the user. For example, as the user completes a signature using a handheld controller of the authentication system 110, the virtual kiosk 600 can display the signature to the user.

[0104] The authentication system 110 can generate an authentication request message containing raw and transformed data collected as the user performs the gesture. The authentication request message can be transmitted to the authentication computer 115 to be compared with a stored authentication model associated with the user.

[0105] For signature authentication, embodiments can measure a similarity between two signatures. Embodiments provide for signature similarity determination that can be divided into two categories 1 ) sequence-based methods and 2) statistical methods.

[0106] For sequence-based methods, the authentication system 110 can process a signature trajectory as a sequence of points or a signal. Each point contains different features, such as location, acceleration, orientation, or even advanced signal from wavelet analysis or FFT. The sequence-based methods can utilize dynamic time warping, cosine distance, longest common subsequence, Euclidean distance, correlation, etc.

[0107] For statistical methods, the authentication system 110 can process a signature as a picture or regards a transformation of a signature signal (like spectrum) as a picture. Then, image processing-based methods can be used such as an auto-encoder for feature extraction and Euclidean distance for similarity measurement, a histogram for feature extraction and KL divergence for similarity measurement, etc.

[0108] At FIG. 6D, if the user is authenticated by the authentication computer 115, the authentication system 110 can receive an authentication response message from the authentication computer 115. The virtual kiosk 600 can display a message indicating the user is authenticated, or in this example, that the pending transaction was completed.

[0109] Disclosed embodiments have several advantages over conventional authentication systems and methods. For example, traditional feature selection techniques in machine learning models use target or proxy classifiers to train a model and determine the importance of each feature, then select features based on the determined importance. Disclosed embodiments describe a hybrid method that is applied to selecting the authentication features (e.g., the raw features and transformed features) based on a combination of statistical methods (mean and standard deviation) and information theory methods (entropy).

[0110] Further, machine learning solutions are designed for user authentications and require training on a large set of user data. This requires the system to use the same set of authentication features for each user. By selecting specific authentication features for each individual user, disclosed embodiments tailor the authentication to the user, which makes the authentication process more robust. Additionally, disclosed embodiments do not involve the use of biometric information from other users and is therefore more privacy preserving. The same is true during authentication itself in that disclosed embodiments only need to rely on the data of the user that is authenticating. Further, disclosed embodiments render spoofing attacks more difficult, as an attacker does not inherently know which features to spoof because the features are unique to each user. Still further, since only a subset of available features is used to authenticate each person, additional computerprocessing is not needed for data associated with features that are less effective for authentication. As such, embodiments provide for systems and methods that reduce required computing resources.

[0111] As described, the inventive service may involve implementing one or more functions, processes, operations or method steps. In some embodiments, the functions, processes, operations or method steps may be implemented as a result of the execution of a set of instructions or software code by a suitably-programmed computing device, microprocessor, data processor, or the like. The set of instructions or software code may be stored in a memory or other form of data storage element which is accessed by the computing device, microprocessor, etc. In other embodiments, the functions, processes, operations or method steps may be implemented by firmware or a dedicated processor, integrated circuit, etc.

[0112] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C++ or Perl using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions, or commands on a computer-readable medium, such as a random access memory (RAM), a read-only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a CD- ROM. Any such computer-readable medium may reside on or within a single computational apparatus, and may be present on or within different computational apparatuses within a system or network.

[0113] While certain exemplary embodiments have been described in detail and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not intended to be restrictive of the broad invention, and that this invention is not to be limited to the specific arrangements and constructions shown and described, since various other modifications may occur to those with ordinary skill in the art.

[0114] As used herein, the use of “a,” “an,” or “the,” is intended to mean “at least one,” unless specifically indicated to the contrary.

Claims

WHAT IS CLAIMED IS:1 . A method comprising: in response to a user initiating a plurality of training gestures, receiving, by a computer, sets of training sensor data instances from a plurality of sensors in an authentication system, the sets of training sensor data instances associated with a set of authentication features; for each set of the sets of training sensor data instances: determining, by the computer, distances between each pair of training sensor data instances in the set of training sensor data instances; and determining, by the computer, a similarity score for the set of training sensor data instances based on the distances; determining, by the computer, a subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances; and creating, by the computer, an authentication model based on the subset of the set of authentication features.

2. The method of claim 1 , further comprising: in response to the user initiating a test gesture, receiving, by the computer, a plurality of test sensor data instances associated with the subset of the set of authentication features; applying, by the computer, the authentication model to the plurality of test sensor data instances; and determining whether or not the user is authenticated based on the authentication model.

3. The method of claim 2, further comprising: determining, by the computer, statistical distributions using the sets of training sensor data instances for each of the authentication features in the subset, andwherein applying the authentication model to the plurality of test sensor data instances, comprises: determining where the plurality of test sensor data instances are located relative to the statistical distributions; determining whether each test sensor data instance in the plurality of test sensor data instances meets a threshold with respect to a statistical distribution in the statistical distributions; and determining if a predetermined number of test sensor data instances meet one or more thresholds with respect to the statistical distributions.

4. The method of claim 2, wherein determining whether or not the user is authenticated based on the authentication model comprises determining whether the plurality of test sensor data instances fall within a pre-defined authentication threshold.

5. The method of claim 2, further comprising: if the user is authenticated based on the authentication model, storing the plurality of test sensor data instances to supplement the sets of training sensor data instances.

6. The method of claim 1 , wherein determining the subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances comprises: determining a mean and a standard deviation of the similarity scores associated with the sets of training sensor data instances for each authentication feature; determining, based on the means and standard deviations of the similarity scores, a variance of the similarity scores associated with the sets of training sensor data instances for each authentication feature; and removing, from the set of authentication features, authentication features having a variance below a variance threshold.

7. The method of claim 6, further comprising:ranking the set of authentication features by the mean, from highest to lowest, associated with the similarity scores associated with each authentication feature of the set of authentication features; ranking the set of authentication features by the variance, from lowest to highest, associated with the similarity scores associated with each authentication feature of the set of authentication features; and removing, from the ranked set of authentication features, a number of authentication features having the lowest mean and the highest variance.

8. The method of claim 1 , wherein determining the subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances comprises: determining an entropy value for each authentication feature in the set of authentication features; ranking the set of authentication features based on their respective entropy value; and removing, from the set of authentication features, a percentage of the ranked set of authentication features associated with the lowest entropy values.

9. The method of claim 1 , wherein the plurality of sensors comprise at least one of a gyroscope and an accelerometer.

10. The method of claim 1 , further comprising: storing, by the computer, the sets of training sensor data instances associated with the subset of authentication features as a signature associated with the user.11 . The method of claim 1 , wherein the plurality of sensors are included in one or more components of the authentication system that enable the user to interact with a virtual environment and wherein the user initiates the plurality of training gestures using the one or more components of the authentication system.

12. The method of claim 11 , wherein the one or more components comprise at least one of: a headset, a handheld controller, and a mobile device.

13. The method of claim 11 , wherein the user initiates the plurality of training gestures by approaching a kiosk in the virtual environment.

14. The method of claim 1 , wherein the set of authentication features comprises at least one feature associated with a physical environment of the user.

15. An authentication computer comprising: a processor; a memory device; and a computer-readable medium coupled to the processor, the computer-readable medium comprising code executable by the processor for implementing a method comprising: in response to a user initiating a plurality of training gestures, receiving, from an authentication system, sets of training sensor data instances from a plurality of sensors of the authentication system, the sets of training sensor data instances associated with a set of authentication features; for each set of the sets of training sensor data instances: determining distances between each pair of training sensor data instances in the set of training sensor data instances; and determining a similarity score for the set of training sensor data instances based on the distances; determining a subset of the set of authentication features based on analyses of the similarity scores associated with the sets of sensor training data instances; and creating an authentication model based on the subset of the set authentication features.

16. The authentication computer of claim 15, wherein the method further comprises:receiving, from the authentication system, an authentication request message comprising a plurality of test sensor data instances associated with the subset of the set of authentication features; applying the authentication model to the plurality of test sensor data instances; determining whether or not the user is authenticated based on the authentication model; and transmitting, to the authentication system, an authentication response message indicating whether or not the user is authenticated.

17. The authentication computer of claim 15, the plurality of training gestures comprises an action of swiping a virtual card at a kiosk in a virtual environment.

18. The authentication computer of claim 15, wherein the plurality of training gestures comprises an action of performing a signature at a kiosk in a virtual environment.

19. The authentication computer of claim 15, wherein the method further comprises: determining whether the set of training sensor data instances meets a threshold amount of data; and if the threshold amount of data is not met, transmitting, to the authentication system, a message to cause the authentication system to prompt the user to repeat the plurality of training gestures.

20. An authentication system comprising: a processor; a plurality of sensors; a memory device; and a computer-readable medium coupled to the processor, the computer-readable medium comprising code executable by the processor for implementing a method comprising:in response to a user initiating a gesture, collecting sets of test sensor data instances from the plurality of sensors, the sets of training sensor data instances associated with a set of authentication features; transmitting, to an authentication computer, an authentication request message comprising the sets of test sensor data instances; and receiving, from the authentication computer, an authentication response message indicating whether the user has been authenticated based on application of an authentication model to the sets of test sensor data instances.

Images