A continuous identity authentication method and system based on VR body interaction

Abstract

The present invention discloses a continuous identity authentication method and system based on VR body interaction, including: obtaining a body movement mode selected by a user; obtaining a virtual viewpoint of a body movement of the user based on virtual movement technology; collecting the user's mobile behavior characteristics based on the movement of the virtual viewpoint; using the mobile behavior characteristics as the user's personal signature, training a time series model based on the mobile behavior characteristics, and obtaining a trained time series model; obtaining the user's mobile behavior characteristics to be authenticated, inputting the mobile behavior characteristics to be authenticated into the trained time series model, and obtaining the confidence of the user's identity prediction; performing threshold judgment on the confidence to obtain the user's identity authentication result. The present invention can continuously monitor the user's identity to prevent the user from being impersonated after logging in, or the user from intentionally leaking the credentials to others.

Description

Technical Field

[0001] The present invention belongs to the technical field of VR device security authentication, and in particular relates to a continuous identity authentication method based on VR body interaction and a system thereof. Background Art

[0002] As virtual reality (VR) technology continues to develop, it is increasingly being used in various fields, such as healthcare, education, gaming, etc. Since a large amount of sensitive data may be generated, it is crucial to ensure that such data is properly protected and to protect the privacy and security of users.

[0003] At present, VR authentication based on traditional PIN codes or passwords is usually a one-time authentication, that is, the user only needs to log in once at the beginning to enter the system, and the user's identity cannot be continuously monitored, which is disadvantageous in some specific application scenarios, such as VR simulation exams or VR e-sports game authentication. At the same time, passwords are also easily leaked or cracked by hackers. In order to solve the limitations of traditional password authentication, many studies have proposed behavior-based authentication methods. Miller et al. can continuously monitor the user's identity during the ball throwing process by identifying the user's behavioral characteristics when throwing the ball in VR, preventing the user from intentionally leaking the password to others, but this authentication method still has limitations. First, this technology has a small scope of application and is only applicable to VR game scenarios with frequent ball throwing. Secondly, this technology is not implicit authentication and cannot be seamlessly connected to other user interactive tasks. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a continuous identity authentication method based on VR body interaction. The present invention can enable users to perform identity authentication seamlessly and continuously during virtual reality movement, and at the same time allow users to perform other interactions while authenticating, so as to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides a continuous identity authentication method based on VR body interaction, comprising:

[0006] Get a limb movement method selected by the user;

[0007] Based on the virtual movement technology, a virtual viewpoint of the user's body movement is obtained; based on the movement of the virtual viewpoint, the user's movement behavior characteristics are collected;

[0008] The mobile behavior feature is used as a personal signature of the user, and a time series model is trained based on the mobile behavior feature to obtain a trained time series model;

[0009] The mobile behavior features to be authenticated of the user are obtained, and the mobile behavior features to be authenticated are input into the trained time series model to obtain the confidence of the user identity prediction; the confidence is threshold judged to obtain the user identity authentication result.

[0010] Preferably, the limb movement modes include: head tilting, arm swinging and knee bending.

[0011] Preferably, the method further includes adjusting the speed of the virtual viewpoint:

[0012] If the user selects the head tilt, the user moves forward in the virtual scene by tilting the head, a first angle between the user's head and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the first angle;

[0013] If the user selects the arm swinging, the user moves forward in the virtual scene by swinging the arm, a second angle between the user's arm and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the second angle;

[0014] If the user selects to bend the knee, the user moves forward in the virtual scene by bending the knee, and a third angle between the user's knee and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the third angle.

[0015] Preferably, the movement behavior characteristics include: the user head display position and the first Euler rotation coordinate, the left and right handle positions and the second Euler rotation coordinate, the knee sensor position and the third Euler rotation coordinate.

[0016] Preferably, the time series model includes: a fully connected neural network and a ShapeNet neural network, wherein based on the fully connected neural network, the confidence of the user identity prediction is output, and based on the ShapeNet neural network, the score value of the user identity prediction is output.

[0017] Preferably, based on the ShapeNet neural network, the process of outputting the score value of the user identity prediction includes:

[0018] We use multi-length input of mobile behavior features to extend the causal convolutional neural network, and use clustering triple loss to uniformly embed and train candidate shapes of different lengths and variables;

[0019] ShapeNet clusters candidate shapes and selects representative and diverse shapelets;

[0020] Using the multivariate Shapelet transform, the distance between the multivariate time series and the selected Shapelet is calculated to obtain the multivariate time series transform representation of the multivariate Shapelet;

[0021] Based on the multivariate time series transformation representation, support vector machine is used for classification and the score value of user identity prediction is output.

[0022] Preferably, the process of obtaining the user's mobile behavior characteristics to be authenticated is as follows:

[0023] Based on the user behavior authentication interface, the user is allowed to select the same limb movement mode to obtain the movement behavior characteristics to be authenticated of the user moving forward in the virtual scene.

[0024] In order to achieve the technical purpose of the present invention, a continuous identity authentication system based on virtual reality body interaction is also provided, characterized in that, for implementing the continuous identity authentication method based on VR body interaction, the system comprises:

[0025] A method acquisition module is used to acquire a limb movement method selected by a user;

[0026] A feature acquisition module is used to obtain a virtual viewpoint of a user's limb movement based on virtual movement technology; based on the movement of the virtual viewpoint, collect the user's movement behavior characteristics;

[0027] A model training module, used to use the mobile behavior characteristics as the user's personal signature, train a time series model based on the mobile behavior characteristics, and obtain a trained time series model;

[0028] The identity authentication module is used to obtain the user's mobile behavior characteristics to be authenticated, input the mobile behavior characteristics to be authenticated into the trained time series model, obtain the confidence of the user identity prediction; perform threshold judgment on the confidence to obtain the user identity authentication result.

[0029] The present invention also provides a readable storage medium, which includes the continuous identity authentication system based on VR body interaction.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The present invention provides a continuous identity authentication method based on virtual reality body interaction, comprising: firstly obtaining a body movement mode selected by a user; then obtaining a virtual viewpoint of a body movement of the user based on virtual movement technology; collecting the user's movement behavior characteristics based on the movement of the virtual viewpoint; using the movement behavior characteristics as the user's personal signature, training a time series model based on the movement behavior characteristics, and obtaining a trained time series model; finally obtaining the user's to-be-authenticated movement behavior characteristics, inputting the to-be-authenticated movement behavior characteristics into the trained time series model, and obtaining the confidence of the user's identity prediction; performing threshold judgment on the confidence, and obtaining the user's identity authentication result.

[0032] The present invention can continuously monitor the identity of the user to prevent the user from being impersonated after logging in, or the user from intentionally leaking the credentials to others. This is particularly useful in some virtual reality simulation examination scenarios.

[0033] The present invention provides an implicit authentication, which does not require the user to provide active identity proof, and can confirm the user's identity by only analyzing the user's movement pattern in a virtual scene.

[0034] The virtual movement proposed by the present invention is widely used in VR scenarios, allowing users to perform mobile authentication while executing other interactive tasks without interrupting other tasks of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 This is a flow chart of a continuous identity authentication method based on VR body interaction according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of a selection interface for a limb movement mode according to an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of collecting user virtual scene movement characteristics according to an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of a system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0041] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Embodiment 1

[0043] like Figure 1 As shown, this embodiment provides a continuous identity authentication method based on VR body interaction, including:

[0044] S1. Obtain a limb movement mode selected by a user;

[0045] Specifically, the user sends a request to set a personal signature (account registration request) to the VR device. When the user enters the system for the first time, he needs to register an account. The user uses the handle to select the "Register Account" button and enters the registered user name. The next step is to enter the mobile action registration interface.

[0046] The VR device receives the request to set a personal signature. At this time, the user can choose from three limb movement methods (head, arm, knee). This is the movement method selection interface, which can be seen in Figure 2 . The movement method of using the head as a limb is that the user moves forward in the virtual world by tilting the head, and the system adjusts the speed of the virtual viewpoint by calculating the angle between the user's head and the horizontal line (for example, y = k * θ, θ is the angle between the head and the horizontal line); the movement method of using the arms as limbs is that the user controls the speed of the virtual viewpoint by swinging the arms, and the larger the arm swing amplitude, the faster the movement speed; the movement method of using the knees as limbs is that the user controls the speed of the virtual viewpoint by bending the knees, and the system adjusts the speed of the virtual viewpoint by calculating the angle between the knees and the horizontal line (for example, y = k * θ, θ is the angle between the knees and the horizontal line).

[0047] S2. Based on virtual movement technology, a virtual viewpoint of the user's body movement is obtained; based on the movement of the virtual viewpoint, the user's movement behavior characteristics are collected;

[0048] Specifically, after the user selects a limb movement mode, the interface jumps to the user movement behavior feature collection interface, such as Figure 3 As shown in the figure, the user is asked to move on a straight line of 30 meters in length. The user presses the button on the handle to start moving, and stops automatically when reaching the end point of the black arrow. During the movement, the position and Euler rotation coordinates of the user's head display, the position and Euler rotation coordinates of the left and right handles, and the position and Euler rotation coordinates of the knee sensor are collected by the device and used for training the time series classifier. The user repeats the above movement 5 times and then ends the feature collection.

[0049] S3, taking the mobile behavior characteristics as the user's personal signature, training a time series model based on the mobile behavior characteristics, and obtaining a trained time series model;

[0050] Specifically, after the user registration is completed, the system will train the user's mobile data features in the virtual reality scene, and use the user's behavioral characteristics during movement as the user's personal signature. After the training process is completed, the system displays that the user has successfully registered and returns to the main menu interface. Model training uses fully connected neural network (FCN) and ShapeNet neural network training. Time series classification model data training model: The core structure of FCN consists of three convolution blocks: the first convolution block contains 128 filters with a convolution kernel size of 8; the second convolution block includes 256 filters with a convolution kernel size of 5; the third convolution block includes 128 filters with a convolution kernel size of 3. They are all batch normalized and activated using the ReLU function. ShapeNet is a deep learning method for shapelet selection for multivariate time series (MTS). Firstly, a multi-length input extended causal convolutional neural network (Mdc-CNN) is used to uniformly embed and train candidate shapelets of different lengths and variables using clustering triplet loss; secondly, instead of directly inputting a large number of candidate shapelets into the classifier, ShapeNet first clusters them and then selects representative and diverse shapelets. Then, the multivariate Shapelet Transform (MST) is used to calculate the distance between the multivariate time series and the selected Shapelet, and the multivariate time series transform representation of the multivariate Shapelet is obtained. Finally, based on the multivariate time series transform representation, support vector machine (SVM) is used for classification.

[0051] S4. Obtain the user's mobile behavior features to be authenticated, input the mobile behavior features to be authenticated into the trained time series model to obtain the confidence of the user identity prediction; perform threshold judgment on the confidence to obtain the user identity authentication result.

[0052] Specifically, S41, the user initiates a user authentication request to the VR device. The user selects the "login button" and enters a user name.

[0053] S42, the mobile VR device receives the user authentication request. At this time, the system jumps to the user behavior authentication interface, and the user is required to move in a straight line with a length of 30m. Figure 3 As shown, similar to the S3 user signature setting stage, the user needs to stop after reaching the red arrow at the end point, and the user's behavior data during the movement process is collected.

[0054] S43. The data of the user's mobile process is input into the model trained in the registration phase. In the fully connected neural network (FCN), the model will output the probability value of each user's identity prediction, where the maximum probability value is the confidence; in the ShapeNet neural network, by calculating the distance between the multivariate time series and the selected Shapelet, and finally classifying it through the support vector machine (SVM), the model will output the score of each user's identity prediction, where the maximum score value is the confidence. If the confidence returned by the model is greater than the set threshold, and at the same time, it is determined that the user name predicted by the model is consistent with the user name entered in the user authentication phase, if so, the user authentication is passed and "Login successful" is displayed; if not, "Login failed, please try again" is displayed. If the confidence returned by the model is less than the set threshold, "Login failed, please try again" is also displayed.

[0055] Beneficial effects of this embodiment:

[0056] This embodiment can continuously monitor the identity of the user to prevent the user from being impersonated after logging in, or the user intentionally leaking the credentials to others. This is particularly useful in some VR simulation test scenarios.

[0057] This embodiment provides an implicit authentication, which does not require the user to provide active identity proof, and the user's identity can be confirmed by simply analyzing the user's movement pattern in the virtual scene.

[0058] The virtual movement proposed in this embodiment is widely used in VR scenarios, allowing users to perform mobile authentication while executing other interactive tasks without interrupting other tasks of the user.

[0059] Embodiment 2

[0060] like Figure 4 As shown, this embodiment provides a continuous identity authentication system based on VR body interaction, which is used to implement the continuous identity authentication method based on VR body interaction described in Example 1. The system includes:

[0061] A method acquisition module is used to acquire a limb movement method selected by a user;

[0062] A feature acquisition module is used to obtain a virtual viewpoint of a user's limb movement based on virtual movement technology; based on the movement of the virtual viewpoint, collect the user's movement behavior characteristics;

[0063] A model training module, used to use the mobile behavior characteristics as the user's personal signature, train a time series model based on the mobile behavior characteristics, and obtain a trained time series model;

[0064] The identity authentication module is used to obtain the user's mobile behavior characteristics to be authenticated, input the mobile behavior characteristics to be authenticated into the trained time series model, obtain the confidence of the user identity prediction; perform threshold judgment on the confidence to obtain the user identity authentication result.

[0065] Embodiment 3

[0066] The present invention also provides a readable storage medium, which includes the continuous identity authentication system based on VR body interaction described in Example 2.

[0067] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A continuous identity authentication method based on VR body interaction, characterized in that: The following steps are involved: Get a limb movement method selected by the user; The limb movement modes include: head tilt, arm swing and knee bending; Based on the virtual movement technology, a virtual viewpoint of the user's limb movement is obtained; based on the movement of the virtual viewpoint, the user's movement behavior characteristics are collected; the movement behavior characteristics include: the user's head display position and the first Euler rotation coordinate, the left and right handle positions and the second Euler rotation coordinate, the knee sensor position and the third Euler rotation coordinate; The mobile behavior characteristics are used as the user's personal signature, and a time series model is trained based on the mobile behavior characteristics to obtain a trained time series model. After the user registration is completed, the system will train the user's mobile behavior characteristics in the virtual reality scene, and use the user's behavior characteristics during movement as the user's personal signature. After the training process is completed, the system will display that the user has successfully registered and return to the main menu interface; The time series model includes: a fully connected neural network and a ShapeNet neural network, wherein based on the fully connected neural network, a first confidence level of user identity prediction is output, and data of the user movement process is input into the model trained in the registration phase, and in the fully connected neural network, the time series model outputs a probability value for each user identity prediction, wherein the maximum probability value is the first confidence level; based on the ShapeNet neural network, a score value for user identity prediction is output, and a second confidence level of user identity prediction is obtained based on the score value, and in the ShapeNet neural network, the time series model outputs a score for each user identity prediction, wherein the maximum score value is the second confidence level; The core structure of the fully connected neural network consists of three convolution blocks: the first convolution block contains 128 filters with a convolution kernel size of 8; the second convolution block contains 256 filters with a convolution kernel size of 5; the third convolution block contains 128 filters with a convolution kernel size of 3; Based on the ShapeNet neural network, the process of outputting the score value of the user identity prediction includes: using a multi-length input extended causal convolutional neural network of mobile behavior features, and using clustering triple loss to uniformly embed and train candidate Shapelets of different lengths and variables; ShapeNet clusters the candidate Shapelets to select representative and diverse Shapelets; using a multivariate Shapelet transform to calculate the distance between the multivariate time series and the selected Shapelet to obtain a multivariate time series transform representation of the multivariate Shapelet; based on the multivariate time series transform representation, using Use support vector machine to perform classification and output the score value of user identity prediction; obtain the user's mobile behavior features to be authenticated, input the mobile behavior features to be authenticated into the trained time series model, and obtain the confidence of user identity prediction; perform threshold judgment on the confidence to obtain the user identity authentication result, wherein, if the confidence returned by the time series model is greater than the set threshold, and at the same time, it is judged that the user name predicted by the model is consistent with the user name entered in the user authentication stage, if so, the user authentication is passed and "Login successful" is displayed; if not, "Login failed, please try again" is displayed, and if the confidence returned by the model is less than the set threshold, "Login failed, please try again" is also displayed; Also included is a process for adjusting the speed of the virtual viewpoint: If the user selects the head tilt, the user moves forward in the virtual reality scene by tilting the head, a first angle between the user's head and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the first angle; If the user selects the arm swinging, the user moves forward in the virtual reality scene by swinging the arm, a second angle between the user's arm and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the second angle; If the user selects to bend the knee, the user moves forward in the virtual reality scene by bending the knee, and a third angle between the user's knee and the horizontal line is calculated, and the speed of the movement of the virtual viewpoint is adjusted according to the third angle.

2. The continuous identity authentication method based on VR body interaction according to claim 1, characterized in that: The process of obtaining the user's mobile behavior characteristics to be authenticated: Based on the user behavior authentication interface, the user is allowed to select a body movement mode to obtain the movement behavior characteristics to be authenticated of the user moving forward in the virtual reality scene.

3. A continuous identity authentication system based on VR body interaction, characterized in that: The system is used to implement the continuous identity authentication method based on VR body interaction according to any one of claims 1 to 2, and comprises: A method acquisition module is used to acquire a limb movement method selected by a user; A feature acquisition module is used to obtain a virtual viewpoint of a user's limb movement based on virtual movement technology; based on the movement of the virtual viewpoint, collect the user's movement behavior characteristics; A model training module, used to use the mobile behavior characteristics as the user's personal signature, train a time series model based on the mobile behavior characteristics, and obtain a trained time series model; The identity authentication module is used to obtain the user's mobile behavior characteristics to be authenticated, input the mobile behavior characteristics to be authenticated into the trained time series model, obtain the confidence of the user identity prediction; perform threshold judgment on the confidence to obtain the user identity authentication result.

4. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the continuous identity authentication method based on VR body interaction as described in any one of claims 1 to 2 is implemented.

Images