Continusous authentication method for user using metaverse service and metaverse service providing device for perfoming continuous user authentication

Abstract

A method for continuous authentication of a user utilizing a metaverse service and a device for providing a metaverse service that performs continuous authentication of the user are disclosed. The disclosed method for continuous authentication of a user utilizing a metaverse service includes the steps of: utilizing the metaverse service and periodically collecting input keys of a user typing on a keyboard, dwell time data on said input keys, and spatial data of the user's hand; inputting said collected data into a pre-trained classification model to classify said collected data into one of a registered user class and an unregistered user class; and periodically performing authentication of said user using said collected data class.

Description

Technology Field

[0001] The present invention relates to a method for continuous authentication of a user, and more specifically, to a method for continuous authentication of a user utilizing a metaverse service and a device for providing a metaverse service that performs continuous authentication of a user. Background Technology

[0003] The term "Metaverse" is a combination of "Meta," a prefix meaning transcendence, and "Universe," referring to the real world. Metaverse technology is garnering attention as a promising technology recognized for its potential to provide innovation across various industries. Driven by this interest, domestic and international big tech companies are consistently developing immersive devices and applications for the metaverse or investing in related ventures, while a diverse range of metaverse services are being developed.

[0004] In particular, as domestic and international companies encouraged remote work during the COVID-19 pandemic, many metaverse office applications, such as Horizon Workrooms, were developed and utilized. Furthermore, even though the pandemic has recently ended, companies continue to use metaverse office applications in the form of hybrid autonomous work as workers reject traditional office work systems.

[0005] However, since metaverse offices store various confidential documents, such as business strategy records, to ensure business continuity without the constraints of time and space, access by malicious attackers like industrial spies could lead to the leakage of these documents and cause massive financial damage. Furthermore, with the emergence of shared office concepts where multiple companies share workspaces and devices, there is a possibility that an employee from a competitor could secretly put on an authenticated immersive device and illegally access confidential documents while an employee from a specific company leaves their desk.

[0006] To prevent such leakage of confidential documents, user authentication and identification technologies in the metaverse have been proposed, but they still have several issues. For example, knowledge-based authentication methods are mostly password-based one-time authentications, making them unsuitable for the metaverse where connection sessions are relatively long; biometric authentication also requires expensive biometric sensors, and privacy concerns are consistently raised. Furthermore, currently proposed behavior-based authentication methods are focused on the gaming sector and require dynamic operations, making them unsuitable for static metaverse office environments.

[0007] Therefore, there is a need for a universal user authentication method that is suitable for metaverse service environments, such as metaverse offices, is economical, and has a low risk of privacy leakage.

[0008] Relevant prior art includes patent documents such as Korean Published Patent No. 2024-0029116, and Korean Registered Patents No. 10-1401656, No. 10-1953165, and No. 10-1597940. The problem to be solved

[0010] The present invention is intended to provide a method and apparatus for performing continuous authentication for a user utilizing a metaverse service.

[0011] Furthermore, the present invention is intended to provide a method and apparatus capable of performing authentication of a user economically and continuously without infringing upon the user's privacy. means of solving the problem

[0013] According to one embodiment of the present invention for achieving the above-mentioned purpose, a method for continuous authentication of a user using a metaverse service is provided, comprising the steps of: using a metaverse service and periodically collecting input keys of a user typing on a keyboard, dwell time data on said input keys, and spatial data of the user's hand; inputting said collected data into a pre-trained classification model to classify said collected data into one of a registered user class and an unregistered user class; and periodically performing authentication of said user using said collected data class.

[0014] In addition, according to another embodiment of the present invention for achieving the above-mentioned purpose, a method for continuous authentication of a user using a metaverse service is provided, comprising: a step of performing a first authentication for a user requesting to use a metaverse service; a step of periodically collecting input keys of a user typing on a keyboard, dwell time data on the input keys, and spatial data of the user's hand according to the result of the first authentication; a step of periodically performing a second authentication for the user using the collected data; and a step of performing a third authentication for the user according to the result of the second authentication.

[0015] In addition, according to another embodiment of the present invention for achieving the above-mentioned purpose, a metaverse service providing device for performing continuous authentication of a user is provided, comprising: a memory; and a processor electrically connected to the memory, wherein the processor utilizes a metaverse service and inputs input keys of a user typing on a keyboard, dwell time data on the input keys, and spatial data of the user's hand into a pre-trained classification model to periodically perform authentication of the user, wherein the classification model is a model that classifies the class of the input data into one of a registered user class and an unregistered user class. Effects of the invention

[0017] According to one embodiment of the present invention, security for a metaverse service can be enhanced through continuous authentication of the user while the user is using the metaverse service.

[0018] In addition, according to one embodiment of the present invention, data collected from the actions of a user typing on a keyboard while using a metaverse service is used for user authentication, thereby enabling continuous user authentication in any metaverse service environment where a keyboard is used, without infringing upon the user's privacy or hindering the user's use of the metaverse service.

[0019] In addition, according to one embodiment of the present invention, continuous authentication of a user can be performed without a separate biometric sensor through a sensor that generates an image of the user's hand. Brief explanation of the drawing

[0021] FIG. 1 is a drawing for explaining a metaverse service providing system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a method for continuous authentication of a user using a metaverse service according to an embodiment of the present invention. Figure 3 is a diagram showing a virtual space for training a classification model. FIG. 4 is a diagram illustrating a method for continuous authentication of a user using a metaverse service according to another embodiment of the present invention. Specific details for implementing the invention

[0022] The present invention is susceptible to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0023] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0025] FIG. 1 is a drawing for explaining a metaverse service providing system according to an embodiment of the present invention.

[0026] Referring to FIG. 1, a metaverse service providing system according to an embodiment of the present invention includes a metaverse service providing device (110) and a user terminal (120).

[0027] The metaverse service providing device (110) provides various metaverse services to the user. For example, the metaverse service provided to the user may be a service that provides a virtual office, i.e., a metaverse office, where the user can work. The user may access the metaverse office and work using a user terminal (120), and the user terminal (120) may be a wearable device such as an HMD or a computing device such as a tablet PC, desktop, or laptop.

[0028] The metaverse service providing device (110) can provide metaverse services to a user after undergoing a user authentication process if the user is a registered user. Additionally, it can continuously perform authentication on the user while the user is using the metaverse service. The metaverse service providing device (110) includes a memory (111) and a processor (120) electrically connected to the memory (111), and the processor (120) can perform a series of processes for continuous authentication of the user. Through such continuous authentication, illegal acts such as unregistered users using the user terminal (120) to steal confidential information while the registered user is away can be prevented.

[0029] The metaverse service providing device (110) can periodically collect data about a user who is using the metaverse service and typing on a keyboard for continuous authentication of the user. This data may include the user's input keys, dwell time data for the input keys, and space data of the user's hand for the input keys.

[0030] Data on the user's input key and dwell time for the input key can be obtained from the keyboard used by the user, and the dwell time data corresponds to the time the user is pressing the input key. In addition, spatial data of the user's hand for the input key can be obtained from an image of the user's hand generated by the camera of the user terminal (120). If the user terminal (120) is an HMD, spatial data of the user's hand for the input key can be generated from an HMD that supports hand tracking.

[0031] A metaverse service providing device (110) can periodically input collected data into a pre-trained classification model to periodically perform authentication of a user, thereby enabling continuous authentication of the user while the user is using the metaverse service. Here, the classification model may be a model that classifies the class of the data input into the classification model into one of a registered user class and an unregistered user class. If the class of the data input into the classification model is classified as a registered user class, user authentication is successful, and if it is classified as an unregistered user class, user authentication fails.

[0032] According to one embodiment of the present invention, security for a metaverse service can be enhanced through continuous authentication of the user while the user is using the metaverse service.

[0033] In addition, according to one embodiment of the present invention, data collected from the actions of a user typing on a keyboard while using a metaverse service is used for user authentication, so that continuous user authentication can be performed in any metaverse service environment where a keyboard is used, without interfering with the user's use of the metaverse service.

[0034] In addition, according to one embodiment of the present invention, continuous authentication of a user can be performed without a separate biometric sensor through a sensor that generates an image of the user's hand.

[0036] FIG. 2 is a diagram illustrating a method for continuous authentication of a user using a metaverse service according to an embodiment of the present invention, and FIG. 3 is a diagram showing a virtual space for learning a classification model.

[0037] A continuous authentication method according to one embodiment of the present invention may be performed on a computing device, and the aforementioned metaverse service providing device is an example of such a computing device.

[0038] Referring to FIG. 2, a computing device according to an embodiment of the present invention performs a first authentication (S210) for a user requesting to use a metaverse service. As an embodiment, the computing device may perform the first authentication using an ID and password entered by the user.

[0039] And, based on the result of the first authentication, the computing device periodically collects input keys of the user typing on the keyboard, data on the dwell time on the input keys, and space data of the user's hand (S220). Then, using the collected data, it periodically performs second authentication for the user (S230).

[0040] As described above, the computing device can perform secondary authentication by inputting collected data into a classification model. The classification model may be a deep learning-based model, and the computing device may provide a virtual space for training the classification model to the user, as shown in FIG. 3. In the virtual space for training the classification model, the computing device provides a script for acquiring training data to the user, and while the user types the provided script using a keyboard, the user's input keys, dwell time data on the input keys, and spatial data of the user's hands may be collected for training the classification model. The collected training data is labeled with a registered user class.

[0041] Then, the computing device performs a third authentication for the user based on the result of the second authentication (S240). If the second authentication fails, the user's use of the metaverse service is blocked, and the third authentication for the user is performed. If the second authentication succeeds, the third authentication is not performed, steps S220 and S230 are repeated, and the second authentication continues to be performed. The third authentication may be performed in the same way as the first authentication or in a different way from the first authentication.

[0042] Meanwhile, according to an embodiment, the computing device may further collect information on jewelry, such as a ring worn on the user's hand, at step S220, and may collect jewelry information through an image of the hand. The user may input jewelry information during the first authentication process, and the computing device may perform second authentication by using the input jewelry information and the collected jewelry information together, as well as the classification model. If the input jewelry information and the collected jewelry information are identical, second authentication may be successful.

[0043] In addition, considering that there may be differences in the spatial data of the user's hand depending on the presence or absence of jewelry, different first and second classification models may be used for secondary authentication depending on the presence or absence of jewelry. For example, the first classification model may be a model trained using training data collected while jewelry is worn, and the second classification model may be a model trained using training data collected while jewelry is not worn. If it is determined through the collected jewelry information that the user is wearing jewelry, the first classification model may be used for secondary authentication, and if it is determined that the user is not wearing jewelry, the second classification model may be used for secondary authentication.

[0045] FIG. 4 is a diagram illustrating a method for continuous authentication of a user using a metaverse service according to another embodiment of the present invention.

[0046] Referring to FIG. 4, a computing device according to an embodiment of the present invention utilizes a metaverse service and periodically collects input keys of a user typing on a keyboard, dwell time data on the input keys, and spatial data of the user's hand (S410). As an example, the spatial data of the user's hand may include at least one of position information, direction information, normal information for at least one joint of the user's hand, and at least one of quaternion information, which is rotation information of the joint. Such spatial data may be spatial data according to the W3C WebXR standard and may include position information, direction information, normal information, and quaternion information for each joint of the wrist, thumb, index finger, middle finger, ring finger, and little finger of the left and right hands. The hands, fingers, joints, etc. used for data collection may vary depending on the embodiment.

[0047] And the spatial data may be spatial data for a virtual hand of a user typing on a virtual keyboard in a metaverse environment, or spatial data for a user's hand of a user typing on a physical keyboard in a physical environment using a metaverse service. A computing device may generate spatial data by calculating position vectors, direction vectors, normal vectors, and quaternion vectors of the user's fingers and wrist joints when the user inputs a key, and mapping the calculated vectors to each input key.

[0048] In this case, according to the embodiment, the spatial data may include absolute spatial data of the first input key and relative spatial data of the second input key with respect to the first input key. The first input key is the first input key among the input keys, and the second input key corresponds to the remaining input keys excluding the first input key. That is, the computing device may generate absolute spatial data for the first input key based on a preset coordinate system, and generate relative spatial data for the second input key based on the absolute spatial data of the first input key.

[0049] For example, if the position vector of the wrist joint for the first input key calculated based on a pre-set coordinate system is [1,1,2] in 3D space and the position vector of the wrist joint for the second input key calculated based on a pre-set coordinate system is [2, 2, 2], then [1, 1, 0], which is the relative position vector of the wrist joint for the second input key calculated from the position vector of the wrist joint for the first input key, can be used as spatial data for the second input key.

[0050] Then, the computing device inputs the data collected in step S410 into a pre-trained classification model to classify the class of the collected data into one of a registered user class and an unregistered user class (S420). Then, using the class of the collected data, authentication for the user is performed periodically (S430).

[0051] In step S430, if the class of the collected data is classified as a registered user class and the probability value for the registered user class is greater than or equal to a preset threshold, the computing device determines the user to be a registered user, and authentication of the user can be successful. That is, the computing device can perform user authentication based on the accuracy of the classification result, and user authentication can be successful if the probability value, i.e., the classification accuracy, is high.

[0052] If the class of collected data is classified as a registered user class, or even if the class of collected data is classified as a registered user class but the probability value for the registered user class is smaller than the threshold, the computing device determines the user as an unregistered user, and authentication for the user fails.

[0053] If user authentication fails in step S430, the computing device may stop providing metaverse services and require additional authentication for the user.

[0055] The technical details described above may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiments, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0057] As described above, the present invention has been explained by specific details such as specific components, limited embodiments, and drawings; however, this is provided merely to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments. A person skilled in the art to which the invention pertains can make various modifications and variations from this description. Therefore, the scope of the invention should not be limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims themselves, shall be considered to fall within the scope of the concept of the invention.

Claims

Claim 1 A method for continuous authentication of a user using a metaverse service, comprising: a step of periodically collecting input keys of a user typing on a keyboard while using a metaverse service, dwell time data on said input keys, and spatial data of the user's hand; a step of inputting said collected data into a pre-trained classification model to classify said collected data into one of a registered user class and an unregistered user class; and a step of periodically performing authentication of said user using said collected data class. Claim 2 A method for persistent authentication of a user using a metaverse service, wherein the spatial data comprises at least one of position information, direction information, normal information, and quaternion information for at least one joint of the user's hand. Claim 3 A method for continuous authentication of a user using a metaverse service, wherein, in claim 2, the spatial data includes absolute spatial data of a first input key and relative spatial data of a second input key with respect to the first input key, the first input key is the first input key among the input keys, and the second input key is the remaining input key among the input keys excluding the first input key. Claim 4 A method for persistent authentication of a user using a metaverse service, wherein, in claim 1, the spatial data is spatial data for a user's virtual hand typing on a virtual keyboard in a metaverse environment. Claim 5 In claim 1, the step of periodically collecting spatial data is a continuous authentication method for a user using a metaverse service, which collects spatial data from an image of the user's hand generated through an HMD worn by the user. Claim 6 A method for continuous authentication of a user using a metaverse service, wherein the step of performing authentication for the user is to determine the user as a registered user when the class of the collected data is classified into the registered user class and the probability value for the registered user class is greater than or equal to a preset threshold. Claim 7 In claim 1, the metaverse service is a service that provides a metaverse office, and the method for continuous authentication of a user utilizing the metaverse service. Claim 8 A method for continuous authentication of a user using a metaverse service, comprising: a step of performing a first authentication for a user requesting to use a metaverse service; a step of periodically collecting input keys of a user typing on a keyboard, dwell time data on the input keys, and spatial data of the user's hand according to the result of the first authentication; a step of periodically performing a second authentication for the user using the collected data; and a step of performing a third authentication for the user according to the result of the second authentication. Claim 9 In claim 8, the step of periodically collecting the spatial data is a continuous authentication method for a user using a metaverse service that further collects information on jewelry worn on the user's hand. Claim 10 A metaverse service providing device for performing continuous authentication of a user, comprising: a memory; and a processor electrically connected to the memory, wherein the processor utilizes a metaverse service and inputs input keys of a user typing on a keyboard, dwell time data on the input keys, and spatial data of the user's hand into a pre-trained classification model to periodically perform authentication of the user, wherein the classification model is a model that classifies the class of the input data into one of a registered user class and an unregistered user class.

Images