Metaverse system and metaverse providing method

Abstract

To use a metaverse anytime and anywhere without depending on an HMD or a place.SOLUTION: The TCP (thin client platform) 30 including the MPF (Metaverse platform) 70,71 that provides the Metaverse space to the head mounted display 10 connected to the network 300 stores user information for user authentication for each user who uses the Metaverse space provided by each Metaverse platform 70,71, performs multi-step user authentication through the thin client authentication server 40, the MPF, and the vendor server that provides the application / service in response to the access of the user from the head mounted display 10, and provides the usage environment of the Metaverse space to the head mounted display when all the multi-step user authentication is successful.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to, for example, a metaverse system and a metaverse providing method. [Background technology]

[0002] When a user uses the metaverse or an XR application (hereinafter referred to as an "XR app"), the user must register an account for the MPF or XR app in advance, log in, and then play.

[0003] Furthermore, when a head-mounted display (hereinafter referred to as HMD) is used with MPF or XR apps, the user environment is stored in the HMD, so the HMD is linked to the platform and XR apps through the user environment.

[0004] In particular, when a metaverse using an HMD is used in corporate activities, each user needs to prepare an HMD and bring it to the location where it will be used. In particular, an avatar for the user's activities needs to be created and registered for each metaverse (Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Special Publication No. 2008-512736 Summary of the Invention [Problem to be solved by the invention]

[0006] As mentioned above, the user environment is stored on the HMD, so the HMD must be carried around to play, which can be inconvenient as it limits the place where you can play. Also, the HMD is tied to the platform of the vendor that provides the app, so you cannot freely access the platform you want to use.

[0007] The present invention has been made in consideration of this background, and aims to provide a metaverse system, a metaverse providing method, and a metaverse providing program that allow users to use the metaverse anytime, anywhere, regardless of HMD or location.

[0008] Furthermore, the present invention aims to provide a metaverse system and a metaverse providing method that allows avatars to be created only once, without the need to create an avatar for each platform or application used. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems and achieve the above-mentioned objectives, one embodiment of the present invention is a metaverse system having one or more metaverse platforms that provides a metaverse space to an HMD connected to a network, and is characterized by comprising: a memory unit that stores user information for user authentication for each user using the metaverse space; an authentication unit that performs multi-stage user authentication for users who use the metaverse space through access from the HMD; and a provision unit that provides a usage environment for the metaverse space to the HMD if all multi-stage user authentications are successful in the authentication unit.

[0010] Another embodiment of the present invention is a metaverse provision method of one or more metaverse platforms that provides a metaverse space to an HMD connected to a network, characterized in that user information for user authentication is stored in memory for each user who uses the metaverse space, and in response to access from the HMD, multi-stage user authentication is performed on the user who uses the metaverse space with that access, and an environment for using the metaverse space is provided to the HMD if all multi-stage user authentications are successful. [Effects of the Invention]

[0011] According to the present invention, it becomes possible to use the metaverse anytime and anywhere, regardless of the HMD or location. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a configuration diagram illustrating an example of the configuration of a metaverse system according to an embodiment of the present invention. [Figure 2] 1 is a configuration diagram showing an example of the configuration of an HMD according to an embodiment of the present invention. [Figure 3] FIG. 1 is a configuration diagram illustrating an example of the configuration of a TCP according to an embodiment of the present invention. [Figure 4] FIG. 2 is a configuration diagram illustrating an example of the configuration of a thin client authentication server according to an embodiment of the present invention. [Figure 5] FIG. 1 is a configuration diagram illustrating an example of the configuration of an MPF ​​authentication server according to an embodiment of the present invention. [Figure 6] FIG. 2 is a configuration diagram illustrating an example of the configuration of a user management server according to an embodiment of the present invention. [Figure 7] FIG. 2 is a configuration diagram illustrating an example of the configuration of a user management server according to an embodiment of the present invention. [Figure 8] FIG. 2 is an explanatory diagram illustrating TCP authentication and application execution according to an embodiment of the present invention. [Figure 9] FIG. 1 is an explanatory diagram illustrating the use of an avatar according to one embodiment of the present invention. [Figure 10] FIG. 2 is an explanatory diagram illustrating a user information DB of the thin client authentication server according to an embodiment of the present invention. [Figure 11] FIG. 2 is an explanatory diagram illustrating a user information DB of an MPF ​​authentication server according to one embodiment of the present invention. [Figure 12] FIG. 3 is an explanatory diagram illustrating a user information DB of the application / service authentication server according to an embodiment of the present invention. [Figure 13] FIG. 3 is an explanatory diagram illustrating a user information DB of a user management server according to an embodiment of the present invention. [Figure 14] 4 is a flowchart illustrating the operation of an MPF ​​according to an embodiment of the present invention. [Figure 15]10 is a flowchart illustrating user authentication and application activation processing of an HMD according to an embodiment of the present invention. [Figure 16] 10 is a flowchart illustrating user authentication of a thin client authentication server according to an embodiment of the present invention. [Figure 17] 10 is a flowchart illustrating another user authentication of the thin-client authentication server according to an embodiment of the present invention. [Figure 18] 10 is a flowchart illustrating user authentication of an MPF ​​authentication server according to an embodiment of the present invention. [Figure 19] 10 is a flowchart illustrating user authentication of an application according to an embodiment of the present invention. [Figure 20] 1 is a flowchart illustrating user authentication from a PC / mobile according to an embodiment of the present invention. [Figure 21] 10 is a flowchart illustrating an application startup process of an HMD according to an embodiment of the present invention. [Figure 22] 10 is a flowchart illustrating an application startup process of an MPF ​​according to an embodiment of the present invention. [Figure 23] 1 is a flowchart illustrating a shared avatar creation and conversion process according to an embodiment of the present invention. [Figure 24] 10 is a flowchart illustrating a common avatar creation process according to an embodiment of the present invention. [Figure 25] 10 is a flowchart illustrating a common avatar conversion process according to an embodiment of the present invention. [Figure 26A] 10 is a flowchart illustrating format conversion during standard avatar conversion according to an embodiment of the present invention. [Figure 26B] FIG. 10 is an explanatory diagram illustrating taste processing according to one embodiment of the present invention. [Figure 27] FIG. 1 is an explanatory diagram illustrating model transformation according to an embodiment of the present invention. [Figure 28] 1 is a flowchart illustrating a taste processing method according to an embodiment of the present invention. [Figure 29]1 is a flowchart illustrating a method for processing the texture of an angular model according to an embodiment of the present invention. [Figure 30] 1 is a flowchart illustrating a method for modifying the texture of a smooth model according to an embodiment of the present invention. [Figure 31] 10 is a flowchart illustrating the reading of avatar data according to an embodiment of the present invention. [Figure 32] 1 is a flowchart illustrating a user file operation according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the following description and drawings are merely examples for explaining the present invention, and some omissions and simplifications have been made as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Furthermore, unless otherwise specified, each component may be singular or plural.

[0014] In the following description, identical or similar components may be designated by the same reference numerals, and redundant description may be omitted. Furthermore, in the following description, various types of information may be described using expressions such as "information" and "table," but the various types of information may also be expressed using other data structures. Furthermore, identification information may be expressed using expressions such as "identification information," "identifier," "name," "ID," and "number," but these are interchangeable. Furthermore, in the following description, "database" will be referred to as "DB," and "table" as "TBL."

[0015] First, the overall system configuration will be described. Fig. 1 is a configuration diagram illustrating an example of the configuration of a metaverse system according to this embodiment. As shown in Fig. 1, the metaverse system is configured such that an HMD (for example, HMD 10) is connected to a thin client platform (hereinafter referred to as TCP) 30 consisting of multiple servers with different functions via a network 300 of the Internet (including wired and wireless communication networks), and end users (hereinafter referred to as users) use avatars to access multiple metaverses.

[0016] The TCP 30 includes a thin client authentication server 40, a user management server 50, and a user environment 60 consisting of a plurality of different metaverse platforms (hereinafter referred to as MPFs). As an example, two MPFs 70 and 71 are shown.

[0017] In this embodiment, assuming corporate use, an example is shown in which a user 200 belonging to a user company 100 uses the metaverse with an HMD 10. Each MPF 70, 71 is connected via a network 300 to vendor servers (two vendor servers 80, 81 are shown as an example) through which a plurality of different vendors provide apps and services.

[0018] In the metaverse system, a user 200 uses an HMD 10, which is a thin client device. Furthermore, when the user 200 uses the metaverse, the user can use MPFs 70 and 71, which are realized by an app based on the user environment set in TCP 30, from any HMD.

[0019] Since the metaverse system allows multiple MPFs 70 and 71 to exist, each MPF 70 and 71 can be accessed from any HMD. Of course, the MPFs that can be accessed are limited to the platform on which the account is registered.

[0020] The HMD 10 is equipped with a thin client application 20 for accessing the TCP 30, and performs user authentication using biometric information or the like to verify the identity of the user and access the user environment registered in the MPF. The MPF to be used for the user environment is set in the TCP 30, and the user can use that MPF by logging in with the HMD 10.

[0021] The avatar that user 200 primarily uses can be shared among all registered MPFs. In this case, by sharing and sharing the avatar data among all MPFs, a single avatar can move between and be active in different MPFs.

[0022] Because the user environment is stored on the server, i.e., on the TCP30 side, content other than user-owned apps (images, videos, etc.) can be shared between MPFs. In other words, the apps and services of the vendors used are built on each MPF and are not installed on the HMD.

[0023] An application that accesses TCP 30 requests user authentication from TCP 30, and after the authentication, the user selects the MPF to use. At the same time, the application sends input information to TCP 30, which transmits it to the application on the MPF and executes it. The application also displays the execution status of applications and services running on the thin client platform 30 on a display. The HMD 10 used by the user 200 only stores an application for accessing the TCP 30, and does not store any user information, application information, etc. In other words, information about the user is stored on the TCP 30, i.e., the user management server 50 side.

[0024] When user 200 uses the system, TCP 30 authentication is first performed using thin client authentication. After that, authentication is performed by each MPF. In this way, the metaverse can only be used once authentication by the thin client and MPF has been passed. After MPF authentication and authentication by the app / service, the app / service can be used. Since there will be many authentications, a single sign-on mechanism using, for example, LDAP (Lightweight Directory Access Protocol) can be used to avoid leaving each authentication to the user.

[0025] The thin client authentication server 40 performs general authentication such as login authentication based on biometric authentication and multi-factor authentication information performed on the HMD side, which is acquired when the user logs in. The user management server 50 sets and registers the user environment and metaverse common avatar of the user who is registered in advance.

[0026] The specific flow of the login operation is as follows: (1) A user 200 logs in to the TCP 30 using the login function (biometric authentication, multi-factor authentication) of the HMD 10. (2) In TCP 30, the thin client authentication server 40 receives user information acquired from the HMD 10 by the login function and performs user authentication. (3) If the user authentication is successful, the thin client authentication server 40 authenticates the user management server 50 and makes it possible to acquire the user environment. (4) The thin client authentication server 40 performs authentication for logging in to the MPF that the user wants to use using the user environment registered in the user management server 50. (5) After successfully logging in to the MPF, if the user uses a vendor-provided app or service, the thin client authentication server 40 also logs in to that app or service using the user environment, etc., registered in the user management server 50.

[0027] In addition, when using MPF and apps / services, the process is as follows: (1) The HMD 10 starts the thin client application 20 and logs in to the MPF. (2) The HMD 10 accesses the user environment through access to the user management server 50 via the TCP 30, and accesses the MPF 70 or 71 used by the user 200. Here, the MPF to be used can be selected by the user's operation of the HMD 10. (3) Once the user 200 has been authenticated for the MPF that he or she wishes to use, that MPF can be used. (4) Furthermore, the user selects an app or service that the user wants to use from the MPF by operating the HMD 10, and the selected app or service is executed as an app execution environment on the MPF. (5) The input information of the user operation is transmitted to the MPF or application / service via the thin client application 20, and the user 200 is ready to operate it. (6) Contents (still images, moving images, etc.) owned by the user 200 are stored in the user management server 50 and can be shared within the TCP 30.

[0028] Next, login will be described with reference to Fig. 2. Fig. 2 is a configuration diagram showing an example of the configuration of an HMD according to this embodiment. For example, as shown in Fig. 2, the HMD 10 includes, as a hardware configuration, a camera 101, a display 102, a communication device 103, an input device 104, a CPU 105, and a memory 106, and, as a software (functional) configuration, an application execution environment 107 including middleware 108, a storage device 109, and an OS (Operating System) 110 of the HMD 10.

[0029] The camera 101 captures the user's retina, iris, etc. to acquire image data, and the display 102 visualizes the execution environment of the TCP 30, such as the metaverse, to display VR (Virtual Reality) images and various operations. The communication device 103 is connected to the network 300 and controls communication with the TCP 30.

[0030] The input device 104 inputs login, metaverse operations, etc. and transmits them to the TCP 30. Although not shown, this input device 104 is capable of wireless communication with an operation remote controller, which is an external device, and acquires input operations through the operation remote controller.

[0031] The CPU 105 controls the entire HMD 10 in accordance with a program stored in the HMD 10. The memory 106 functions as a work area during control by the CPU 105, and also temporarily caches image data of the eyes captured by the camera 101 for use during authentication.

[0032] The middleware 108 is composed of a user authentication processing unit 121, an application management unit 122, and a communication unit 123. The user authentication processing unit 121 requests authentication from the thin client authentication server 40 based on image data of the retina, iris, etc. captured by the camera 101 and user information, and obtains permission for use if the authentication is successful.

[0033] The application management unit 122 manages the execution of the thin client application 20, and the communication unit 123 controls the overall communication in cooperation with the communication device 103. The storage device 109 stores the thin client application 20 and the like. The thin client application 20 is an application that connects to the user environment of the TCP 30, and processes input and output through communication with the TCP 30.

[0034] Next, the TCP 30 will be described with reference to Fig. 3. Fig. 3 is a configuration diagram illustrating an example of the configuration of the TCP 30 according to this embodiment. For example, as shown in Fig. 3, the TCP 30 is configured by a thin client server 31, a communication device 32, an OS 39 of the TCP 30, a thin client authentication server 40, a user management server 50, an avatar processing unit 33, and MPFs 70 and 71.

[0035] The thin client server 31 executes a server function for communicating with the client-side HMD 10, and the communication device 32 manages communication with the HMD 10 and other communication devices via the network 300.

[0036] The thin client authentication server 40 executes the above-mentioned authentication process by the user authentication processing unit 460, and the user management server 50 executes the above-mentioned user management by the user management unit 501.

[0037] The avatar processing unit 33 includes an avatar creation unit 301 and an avatar conversion unit 302. The avatar creation unit 301 creates an avatar base (image data) to be used in common across all MPFs via a device such as the HMD 10, and registers it in the user management unit 501.

[0038] The avatar conversion unit 302 converts the avatars registered in the user management unit 501 to conform to the specifications of each MPF. If the image data of the avatar is formed using a polygon mesh (hereinafter referred to as a mesh) of a predetermined granularity, this conversion includes a process of adjusting the granularity of the mesh to conform to the MPF specifications. Of course, in addition to the granularity of the mesh, it is also possible to appropriately take into account the color scheme, texture, and other factors. The shape of the mesh is triangular.

[0039] Furthermore, for avatars, for example, a deformed human shape is preferable in terms of corporate usage, and it is preferable that the user can be easily recognized in any MPF. Of course, users can create multiple base avatars, and select one by operating the HMD 10 when using the Metaverse.

[0040] The MPF 70 includes an MPF ​​authentication server 701 and an application / service provision unit 703 that provides applications / services to the HMD 10. The MPF authentication server 701 authenticates user use of the platform using an MPF ​​authentication unit 702. The application / service provision unit 703 is an environment that executes applications / services in the user usage environment shown in FIG. 8, which will be described later.

[0041] Next, the thin client authentication server 40 will be described with reference to Fig. 4. Fig. 4 is a configuration diagram illustrating an example of the configuration of the thin client authentication server 40 according to this embodiment. For example, as shown in Fig. 4, the thin client authentication server 40 is configured by an input device 401, an output device 402, a CPU 403, a memory 404, a communication device 405, middleware 406, a storage device 407, an OS 408 of the thin client authentication server 40, and the like.

[0042] For user authentication, the input device 401 inputs biometric information (images of the retina, iris, etc.) and multi-factor authentication information set in advance by the user from the HMD 10. When user authentication is successful, the output device 402 outputs the user information to the user management server 50 and MPF, leading to use of the metaverse.

[0043] The CPU 403 controls the entire thin client authentication server 40, and the memory 404 is used as a work area for storing various data during execution of the CPU 403. The communication device 405 communicates with the HMD 10, the MPFs 70 and 71, and the MPF authentication server 701 to perform authentication exchanges.

[0044] The middleware 406 includes a user authentication processing unit 460, a user management unit 463, and a communication unit 464. The user authentication processing unit 460 includes a biometric authentication processing unit 461 and a multi-factor authentication processing unit 462.

[0045] The biometric authentication processing unit 461 and the multi-factor authentication processing unit 462 communicate via the communication unit 464, and compare the biometric information (images of the retina, iris, etc.) received and input from the HMD10 with the multi-factor authentication information to perform authentication of the thin client (user authentication).

[0046] This user authentication result is stored and managed in association with the user by the user management unit 463. After the user authentication, the user can log in to the MPF or vendor's application / service using SSO (Single Sign-On) such as LDAP.

[0047] The user management unit 463 registers user information 470-1 to 470-N in the storage device 407 and provides the user information 470-1 to 470-N to the user authentication processing unit 460. The communication unit 464 cooperates with the communication device 405 and handles communication required for user authentication.

[0048] The storage device 407 stores user information 470-1 to 470-N (N is a natural number) for users who have registered accounts. Taking user information 470-1 as an example, the user information stores personal information 471 such as name, contact information, and payment, biometric information 472 registered via the HMD 10, multi-factor authentication information 473 set via the HMD 10 or a computer such as a PC or mobile terminal, and registered platform information 474 indicating the MPF used by the user. Note that this user information can be changed as appropriate by user operation after user authentication.

[0049] Next, the MPF authentication server 701 will be described using Fig. 5. Fig. 5 is a configuration diagram illustrating an example of the configuration of the MPF authentication server 701 according to this embodiment. As shown in Fig. 5, the MPF authentication server 701 is composed of an input device 711, an output device 712, a CPU 713, a memory 714, a communication device 715, middleware 716, a storage device 717, and an OS 718 of the MPF authentication server 701, for example.

[0050] The input device 711 inputs user information for platform authentication for using the metaverse from the thin client authentication server 4. When the platform authentication is successful, the output device 712 performs processing such as outputting the platform authentication result to the thin client authentication server 4.

[0051] The CPU 713 controls the entire MPF authentication server 701, and the memory 714 is used as a work area for storing various data during execution of the CPU 713. The communication device 715 communicates with the thin client authentication server 4 to execute authentication exchanges.

[0052] The middleware 716 includes an MPF ​​authentication unit 702, which is a platform-side authentication unit, an application management unit 723, and a communication unit 724. The MPF authentication unit 702 includes an authentication processing unit 721 that executes platform authentication by comparing user information received from the thin client authentication server 40, and a user management unit 722 that performs user management based on the user information described below.

[0053] The application management unit 723 manages applications corresponding to applications / services that can be executed by the MPF. The communication unit 724 handles communication related to platform authentication in cooperation with the communication device 715. This communication unit 724 communicates with the HMD 10 and an application / service authentication server (described later) to perform authentication exchanges.

[0054] Storage device 717 of the platform-side storage unit stores user information 770-1 to 770-N (N is a natural number) for users who have registered accounts. Taking user information 770-1 as an example, the user information stores personal information 771 such as name, contact information, and payment, authentication information 772 such as an account for the user to authenticate to the platform, and registered app / service information 773 indicating apps / services available to the user. Note that this user information can be changed as appropriate by user operation after user authentication.

[0055] Next, the application / service authentication server provided in the vendor servers 80 and 81 will be described with reference to Fig. 6. Fig. 6 is a configuration diagram illustrating an example of the configuration of the application / service authentication server according to this embodiment. For example, as shown in Fig. 6, the application / service authentication server 801 is configured with an input device 811, an output device 812, a CPU 813, a memory 814, a communication device 815, middleware 816, a storage device 817, and an OS 818 of the application / service authentication server 801.

[0056] The input device 811 inputs user information for authentication of use of this application / service from the MPF authentication server 701. When user authentication is successful, the output device 812 performs processing such as outputting (responding to) the user authentication result to the MPF authentication server 701.

[0057] The CPU 813 controls the entire application / service authentication server 801, and the memory 814 is used as a work area for storing various data during execution of the CPU 813. The communication device 815 communicates with the MPF authentication server 701 to execute authentication transactions.

[0058] The middleware 816 includes a user authentication processing unit 821, a user management unit 823, and a communication unit 824. The user authentication processing unit 821 of the vendor authentication unit compares the user information received from the MPF authentication server 701, executes authentication for use of the application / service, and returns the result to the MPF authentication server 701. If this authentication is successful, user authentication (launch permission) for available applications / services is executed.

[0059] The user management unit 823 registers user information 870-1 to 870-N in the storage device 817 and provides the user information 870-1 to 870-N to the user authentication processing unit 821. The communication unit 824, in cooperation with the communication device 815, is responsible for communication required for user authentication with the thin client authentication server 40 and the MPF authentication server 701.

[0060] Storage device 817 of the vendor-side storage unit stores user information 870-1 to 870-N (N is a natural number) for users who have registered accounts. Taking user information 870-1 as an example, the user information stores personal information 871 such as name, contact information, and payment, and registered app / service information 872 indicating apps / services available to the user. Note that this user information can be changed as appropriate by user operation after user authentication.

[0061] Next, the user management server 50 will be described with reference to Fig. 7. Fig. 7 is a configuration diagram illustrating an example of the configuration of the user management server 50 according to this embodiment. For example, as shown in Fig. 7, the user management server 50 is configured with an input device 511, an output device 512, a CPU 513, a memory 514, a communication device 515, middleware 516, a storage device 517, and an OS 518 of the user management server 50.

[0062] The input device 511 inputs user information for user authentication from the MPF authentication server 701. The output device 512 performs processing such as outputting (responding to) the user authentication result to the MPF authentication server 701 when user authentication is successful.

[0063] The CPU 513 controls the entire user management server 50, and the memory 514 is used as a work area for storing various data during execution of the CPU 513. The communication device 515 communicates with the thin client authentication server 40 and other servers to execute authentication transactions.

[0064] The middleware 516 includes a user authentication processing unit 521, a user management unit 523, and a communication unit 524. The user authentication processing unit 521 executes user authentication by comparing the user information received from the thin client authentication server 40, and returns the result to the thin client authentication server 40. If this authentication is successful, the process proceeds to the next authentication step.

[0065] The user management unit 523 registers user information 570-1 to 570-N in the storage device 517 and provides the user information 570-1 to 570-N to the user authentication processing unit 521. The communication unit 524 cooperates with the communication device 515 and handles various communications with the thin client authentication server 40 and other servers.

[0066] The storage device 517 stores user information 570-1 to 570-N (N is a natural number) for users who have registered accounts. For example, user information 570-1 stores personal information 571 such as name, contact information, and payment information, and various user data 572 for using the metaverse.

[0067] Regarding user data, files created by the user themselves are stored and linked to the user. In particular, avatars created by users are composed of data common to the entire metaverse, and the format can be converted depending on the MPF used. Furthermore, this user information can be changed as needed by the user after user authentication.

[0068] Next, the operation of TCP 30 will be described with reference to Fig. 8. Fig. 8 is an explanatory diagram illustrating authentication and application execution of TCP 30 according to this embodiment. In Fig. 8, arrows with diagonal lines indicate the flow of authentication processing, and arrows without patterns indicate the flow of application execution.

[0069] Regarding the flow of authentication processing, first-stage user authentication (first user authentication) is performed by the thin client application 20 of the HMD 10 at the thin client authentication server 40 of the thin client server 31, and then second-stage user authentication (second user authentication) is performed at the MPF authentication server 701 of the MPF 70 or 71. When using an app / service, third-stage user authentication (third user authentication) is further performed at the app / service authentication server 801 of the vendor server 80 or 81.

[0070] Regarding the flow of application execution, the thin client application 20 of the HMD 10 connects to the user environment of the MPF 70 or 71 and receives the application / service. Regarding access to user data, the user environment receives data from the user management server 50 according to the application / service of the MPF 70 or 71.

[0071] The thin client application 20 of the HMD 10 connects to the thin client server 31 and connects to an MPF ​​(for example, MPF 70 or 71) constructed as a thin client. The thin client server 31 transmits the execution status of the application / service being executed in the connected MPF to the thin client application 20 of the HMD 10 by streaming communication.

[0072] The thin client application 20 of the HMD 10 displays on the display the execution status received from the thin client server 31. When the user inputs an operation for an application or a service, the input information is used by the thin client server 31 and the thin client application 20 in cooperation to execute (operate) the application or service.

[0073] Next, the use of avatars will be described with reference to Fig. 9. Fig. 9 is an explanatory diagram illustrating the use of avatars according to this embodiment. In Fig. 9, arrows with a grid pattern indicate the flow of avatars, and arrows without a pattern indicate the flow of application execution.

[0074] TCP30 provides an avatar creation function. Avatars used in apps / services are owned by each app or service. A common avatar (common avatar) is created on the thin client and saved as an avatar image. This avatar image is used as a base for conversion to avatars appropriate for various metaverses.

[0075] The avatar is a common avatar for thin clients, and it is possible to create avatars specific to MPF and apps / services. Avatars are created on the thin client side, stored as user data, and registered in MPF, apps, and services.

[0076] Apps / services are executed by selecting whether to use a common avatar or a uniquely created avatar. The avatar can be used on the HMD 10, PC, and mobile device with the same account. The common avatar and the uniquely created avatar are registered in the user management server 50. Vendor apps and services provided by the MPFs 70 and 71 use the common avatar or unique avatar registered in the user management server 50, and the common avatar is converted to fit the metaverse.

[0077] To summarize the avatar flow, for a common avatar, the thin client application 20 of the HMD 10 accesses the thin client server 31 to create a common avatar and create an avatar according to each MPF, and convert the shared avatar.

[0078] Next, various DBs will be described with reference to Fig. 10 to Fig. 13. Fig. 10 is an explanatory diagram illustrating the user information DB of the thin client authentication server 40 according to this embodiment, Fig. 11 is an explanatory diagram illustrating the user information DB of the MPF authentication server 701 according to this embodiment, Fig. 12 is an explanatory diagram illustrating the user information DB of the application / service authentication server 801 according to this embodiment, and Fig. 13 is an explanatory diagram illustrating the user information DB of the user management server 50 according to this embodiment.

[0079] 10, the thin client authentication server 40 has a user information DB 407A set in the storage device 407. This user information DB 407A stores the user information 407-1 to 407-N described in Fig. 4. To explain the user information 407-1 in more detail, authentication information 407-12 and various user information 407-13 are stored in association with a user ID 407-11 that identifies the user.

[0080] The authentication information 407-12 includes biometric information 472, multi-factor authentication information 473, password, etc., and the various user information 407-13 stores personal information 471 such as gender and age, language, and registered platform information 474. Note that the user ID 407-11 can identify an individual by linking it with the information in the authentication information 407-12, and is therefore considered to correspond to the personal information 471 here.

[0081] In the authentication information 407-12, only the minimum information required for authentication processing is described as information managed by the thin client authentication server 40. As for the information in the biometric information 472, biometric information of the user (such as the retina and iris of the eye) is managed.

[0082] Multi-factor authentication information 473 manages a set of input method patterns (image selection, lock pattern, PIN, email authentication, etc.) and information on the correct answer for that pattern. User authentication information 470-12 is shared by the MPF and application / service vendors. Various user information 407-13 is information managed by having the user enter required information. The required information differs depending on the MPF, and various user information 470-13 is user information required by the MPF.

[0083] As shown in FIG. 11, the MPF authentication server 701 has a user information DB 717A set in the storage device 717. This user information DB 717A stores the user information 770-1 to 770-N described in FIG. 5. To explain the user information 770-1 in more detail, authentication information 72 and registered application / service information 773 are stored in association with a user ID 770-11 that identifies the user. The authentication information 772 includes biometric information, multi-factor authentication information, a password, and the like. Note that the user ID 770-11 can identify an individual in association with other information, and therefore is considered to correspond to personal information 771 here.

[0084] The user ID 770-11 is the ID of the user who registered the account. The registered application / service information 773 is information about which application or service the target user is using (registering), and the user authentication information 772 is information shared with the thin client and application / service vendor.

[0085] 12, application / service authentication server 801 has user information DB 817A set in storage device 817. This user information DB 817A stores user information 870-1 to 870-N described in FIG. 6. To explain user information 870-1 in more detail, registered application / service information 872 is stored in association with user ID 870-11 that identifies the user. Note that user ID 870-11 can identify an individual in association with other information, and therefore is considered to correspond to personal information 871 here.

[0086] The user ID 870-11 is the ID of the user who registered the account. The registered application / service information 872 is information indicating which application or service the target user is using (registered). In addition, the user authentication information is shared with the thin client and MPF, and the application / vendor basically depends on the database of each vendor.

[0087] 13, the user management server 50 has a user information DB 517A set in the storage device 517. This user information DB 517A stores the user information 570-1 to 570-N described in FIG. 7. To explain the user information 570-1 in more detail, the user information 570-1 is stored as user data 572 linked to a user ID 570-11 that identifies the user. Note that the user ID 570-11 can identify an individual in connection with other information, and therefore is considered to correspond to personal information 571 here.

[0088] The user ID 570-11 is the ID of the user who registered the account. The user data 572 is a directory divided for each user, and is data such as files and avatar files. Authentication is performed using single sign-on when logging in to TCP. The assigned directory can be used freely by the user, and the root directory cannot be deleted.

[0089] Next, the authentication operation of the MPF will be described with reference to Fig. 14 to Fig. 20. Fig. 14 is a flowchart illustrating the operation of the MPF according to this embodiment, Fig. 15 is a flowchart illustrating user authentication of the HMD and application startup processing according to this embodiment, Fig. 16 is a flowchart illustrating user authentication of the thin client authentication server according to this embodiment, Fig. 17 is a flowchart illustrating other user authentication of the thin client authentication server according to this embodiment, Fig. 18 is a flowchart illustrating user authentication of the MPF authentication server according to this embodiment, Fig. 19 is a flowchart illustrating user authentication of an application according to this embodiment, and Fig. 20 is a flowchart illustrating user authentication from a PC / mobile device according to this embodiment.

[0090] Fig. 14 shows the processing by the thin client server 31 of the TCP 30 shown in Fig. 3. When it is confirmed that the HMD 10 is powered on (step S1401), thin client authentication processing is executed (step S1402), and thin client avatar creation processing is executed (step S1403).

[0091] Next, the MPF to be used by the user is selected (step S1404), and authentication processing for that MPF is executed (step S1405). Furthermore, an avatar to be used with that MPF is created and registered (step S1406).

[0092] Once the avatar is ready in this way, the user can start using the application / service (step S1407). First, authentication processing for the application / service is performed (step S1408), and an avatar for the application / service is created (step S1409). Then, the application / service is executed on the MPF (step S1410).

[0093] When the HMD 10 is powered on (step S1501), the thin client application 20 is started (step S1502), and user authentication is performed in cooperation with the user authentication processing unit 460 (step S1503). Here, the camera 101 photographs the user's eyes (step S1504), and the photographed biometric information is transmitted to the thin client authentication server 40 (step S1505).

[0094] The user authentication processing unit 460 of the thin client authentication server 40 executes authentication processing (step S1506), and then executes avatar creation processing for the thin client (step S1507). The processing of step S1506 will be described in detail with reference to FIG. 16, and the processing of step S1507 will be described in detail with reference to FIG.

[0095] If the user authentication is successful in step S1506 (YES route in step S1508), the user authentication result and the MPF list to be used are acquired (step S1509). On the other hand, if the user authentication is not successful (NO route in step S1508), the device is deemed unavailable and the process ends (step S1517).

[0096] Next, an MPF ​​to be used is selected (step S1510), and the MPF authentication server 701 performs user authentication (step S1511). The processing of step S1511 will be described in detail in FIG. 17. If the user authentication is successful (YES route in step S1512), a connection to the MPF to be used is made (step S1513), and an MPF ​​avatar creation / conversion process is performed (step S1514). An application to be used is then selected (step S1515), and an application avatar creation / conversion process is performed therein (step S1516). Steps S1513 and S1516 will be described in detail in FIG. 25. Step S1515 will be described in detail in FIG. 21.

[0097] Furthermore, if user authentication cannot be performed in step S1512 (NO route in step S1512), the service is deemed unavailable and the process ends (step S1517). Note that the determination of whether the service is unavailable may be made unavailable (user locked) if an error occurs a certain number of times, or a user without a registered account may be deemed an unregistered user and therefore unavailable, but is not limited to these.

[0098] Next, the authentication process in the thin client authentication server 40 in step S1506 described above will be described with reference to Fig. 16. First, user information is received from the HMD 10 (step S1601). The user authentication processing unit 460 searches the storage device 407 for information that matches the received user information (step S1602), and performs user authentication based on the search result (step S1603).

[0099] If user authentication is not successful (NO route in step S1604), a notification of authentication failure is sent to the HMD 10, and this process ends (step S1605). On the other hand, if user authentication is successful (YES route in step S1604), information on available MPFs (MPF information) is acquired by the user management unit 523 of the user management server 50 based on the user information (step S1606). Furthermore, the authentication result in step S1604 and the MPF information are sent to the HMD 10 (step S1607).

[0100] User authentication involves both biometric authentication and multi-factor authentication via user input. The user can select the multi-factor authentication pattern from methods such as selecting an image, password, PIN number, or lock pattern. When selecting an image, the user registers an image prepared in advance by the authentication function or an image selected by the user from images stored by the user. In this case, the user must register the image via a PC or smartphone.

[0101] Next, the execution of MPF authentication in step S1511 will be described with reference to Fig. 17. When MPF ​​information is received from the HMD 10 (step S1701), the user management unit 463 acquires MPF account information (step S1702), and the user authentication processing unit 460 requests authentication processing from the MPF (step S1703).

[0102] Then, authentication processing of the MPF authentication server 701 is executed (step S1704). This step S1704 will be described in detail with reference to FIG. 18. If authentication is successful in step S1704 (YES route in step S1705), a notification of authentication success is sent to the HMD 10 (step S1706). On the other hand, if authentication is not successful in step S1704 (NO route in step S1705), a notification of authentication failure is sent to the HMD 10 (step S1707). In this way, this processing ends.

[0103] Next, the authentication process in step S1704 will be described with reference to Fig. 18. When the MPF authentication server 701 receives user information from the thin client server 17 (step S1801), the user information DB 717A is referenced and target user information corresponding to the received user information is acquired (step S1802), and the authentication processing unit 721 executes user authentication processing (step S1803).

[0104] If the user authentication is successful (YES route in step S1804), a notification of successful authentication is sent to the HMD 10 (step S1805), whereas if the user authentication is successful (NO route in step S1804), a notification of unsuccessful authentication is sent to the HMD 10 (step S1806). In this manner, this process ends.

[0105] Next, user authentication for an application / service, which corresponds to step S1408, will be described with reference to Fig. 19. When the application / service authentication server 801 receives user information from the MPF authentication server 701 (step S1901), it references the user information DB 817A and then acquires target user information corresponding to the received user information (step S1902). Next, the user authentication processing unit 821 executes user authentication processing (step S1903).

[0106] If the user authentication is successful (YES route in step S1904), a notification of successful authentication is sent to the HMD 10 (step S1905), whereas if the user authentication is successful (NO route in step S1904), a notification of unsuccessful authentication is sent to the HMD 10 (step S1906). In this manner, this process ends.

[0107] Next, the authentication process from a PC / mobile device will be described with reference to Fig. 20. In the thin client server 31, first, a user logs in from an access terminal (a PC or a mobile device such as a smartphone) and when the login information is accepted (step S2001), a request for user authentication is sent to the thin client authentication server 40 based on the login information (step S2002). The login is performed using multi-factor authentication from the access terminal.

[0108] If the user authentication is successful (YES route at step S2003), a notification of successful authentication is sent to the access terminal (step S2004), whereas if the user authentication is successful (NO route at step S2003), a notification of unsuccessful authentication is sent to the access terminal (step S2005). This is how the process ends.

[0109] Next, application startup will be described with reference to Fig. 21 and Fig. 22. Fig. 21 is a flowchart illustrating application startup processing of the HMD according to this embodiment, and Fig. 22 is a flowchart illustrating application startup processing of the MPF according to this embodiment.

[0110] Regarding application activation, first, available applications are displayed on the display 102 of the HMD 10 in a selectable manner, and application selection is performed by the user operating the input device 104 (step S2101). When an application is selected, a request to activate the application is made to the MPF (step S2102), and the MPF executes application activation processing, which will be described in detail in Fig. 22 (step S2103).

[0111] If application startup is permitted in the above application startup process (YES route in step S2104), the application is started (step S2105), and processing to display the application screen on the HMD 10 is executed (step S2106). On the other hand, if application startup is not permitted (NO route in step S2104), application startup is disabled (step S2105), and processing to display the application screen on the HMD 10 is executed (step S2106). In this manner, this processing ends.

[0112] 22, in the above-described application activation process, when the thin client server 31 receives an application activation request from the HMD 10 (step S2201), it transmits a user authentication request to the application / service authentication server 801 (step S2202). The application / service authentication server 801 executes the user authentication process of FIG. 19 (step S2203) and receives the authentication result (step S2204).

[0113] If the received authentication result confirms that the user authentication is successful (YES route at step S2205), the application launch flag is set to ON, indicating that the application is allowed to launch (step S2206), and if the user authentication is confirmed to be unsuccessful (NO route at step S2205), the application launch flag is set to OFF, indicating that the application is not allowed to launch (step S2207). In this way, this process ends.

[0114] Next, avatar-related processing will be described with reference to Fig. 23 to Fig. 30. Fig. 23 is a flowchart illustrating common avatar creation and conversion processing according to this embodiment, Fig. 24 is a flowchart illustrating common avatar creation processing according to this embodiment, and Fig. 25 is a flowchart illustrating common avatar conversion processing according to this embodiment. Note that the definition of an avatar is that an avatar created by a thin client is a common avatar, and an avatar provided by an application or service is a standard avatar.

[0115] In the MPF, first, user authentication is performed (step S2301), and if the authentication is successful, the user management unit accepts the user's access (step S2302). Because it is a thin client, authentication is performed automatically when accessing the server. Then, when user information is acquired by the user management unit (step S2303), a common avatar creation process, which will be described in detail in FIG. 24, is executed (step S2304). Then, an available MPF is selected by user operation (step S2305), and an application launch request is sent to the selected MPF (step S2306). First, an MPF ​​avatar creation and registration process, which will be described in detail in FIG. 25, is executed (step S2307).

[0116] Next, an application / service is selected in response to a user operation (step S2308), and the selected application / service is launched (step S2309). In cases where an application uses an avatar, an avatar for the application is created and registered (step S2310). After this, the application's avatar, which will be described in detail in FIG. 25, is also prepared, and the launched application / service is executed (step S2311). For applications that use avatars, a function for selecting an avatar to be used is provided as a function of the application.

[0117] In the common avatar creation process in step S2304, as shown in FIG. 24, when user information is acquired in the user management unit (step S2401), an operation as to whether or not to create an avatar for the user is accepted (step S2402). If an operation as to not create an avatar is accepted (NO route in step S2402), this process ends.

[0118] If an operation to create an avatar is accepted (YES route in step S2402), a common avatar is created in response to the user's operation (step S2403), and the data of the created common avatar is registered (saved) (step S2404). This avatar creation includes editing of an avatar that has already been created. This is how the process ends.

[0119] In addition, the MPF avatar creation and registration process in step S2307 first determines whether avatar conversion is necessary (step S2501), as shown in Fig. 25. It is also assumed that multiple metaverses apply a common format, and as an example, avatar conversion is necessary when the formats are different.

[0120] If it is determined that avatar conversion is not necessary (NO route in step S2501), the process proceeds to step S2507, where the avatar data is handed over to the caller via an API or the like, and the process ends.

[0121] On the other hand, if the determination result indicates that avatar conversion is necessary (YES route in step S2501), the avatar format conversion process detailed in Figures 26A and 26B is executed (step S2502). Then, the converted avatar is displayed and presented to the user (step S2503), and approval is accepted by the user (step S2504).

[0122] If the approval operation for the converted avatar is accepted (YES route in step S2504), the converted avatar is output in the specified format (step S2506), and the avatar data is delivered to the caller via an API or the like (step S2507). On the other hand, if the approval operation for the converted avatar is not accepted (NO route in step S2504), the avatar is manually edited by the user (step S2505), and the process returns to step S2504, where the same process is repeated.

[0123] The above functions are provided as functions that can be accessed from MPF, apps / services, etc. Avatar data stored as user data on the thin client can be accessed via URL, API, etc., and the created avatar data can be handed over. Tastes can also be specified via the API.

[0124] Next, avatar processing will be described using Figures 26A to 30. Figure 26A is a flowchart illustrating format conversion during standard avatar conversion according to this embodiment, Figure 26B is an explanatory diagram illustrating taste processing according to this embodiment, Figure 27 is an explanatory diagram illustrating model conversion according to one embodiment of the present invention, Figure 28 is a flowchart illustrating a taste processing method according to this embodiment, Figure 29 is a flowchart illustrating a taste processing method for an angular model according to this embodiment, and Figure 30 is a flowchart illustrating a taste processing method for a smooth model according to this embodiment.

[0125] Regarding format conversion, as shown in Fig. 26A, first, avatar data created on a thin client is read (step S2601), and the data structure is parsed as binary data (step S2602). A conversion process to a format specified for conversion to the metaverse to be used is selected (step S2603).

[0126] Then, the various property values ​​and data components of the thin client avatar are read (step S2604). In the parser process, the binary data and data structure are read in order and the process is repeated. Note that a process is prepared for each format to be converted.

[0127] The property values ​​and data components read above are set for each data structure in the destination format, and taste processing is then performed (step S2605). If a taste is specified, the avatar's atmosphere is processed to match that taste. For example, in the case of a deformed taste, the avatar's head-to-body ratio is calculated to be about half of the total body, and the avatar's physique is processed to be rounded.

[0128] Thereafter, standard property values ​​of the post-conversion format are set (step S2606). The property values ​​are set to standard setting values ​​that are not present in the thin client avatar but are present only in the conversion destination format.

[0129] If a taste is specified, the atmosphere of the avatar is modified to match the taste, as shown in Fig. 26B. Two examples of tastes are given: an angular taste and a rounded taste.

[0130] The above two tastes and how to process them are stored in the avatar processing unit 33. For example, in the case of a deformed taste, it is best to set the parameter indicating the taste value to "1," calculate the avatar's head-to-body ratio to about half of the total body, and process the avatar's physique to be rounded. In addition, in the case of a polygon taste, it is best to set the parameter indicating the taste value to "2," reduce the number of avatar meshes, and increase one to process it into an angular model. In this way, it is possible to process it into a preferred taste by adjusting the mesh.

[0131] The process involves obtaining the avatar format that works with the MPF and app from the platform and app. It is assumed that the format will be called from the MPF and app. When making the call, the conversion format is specified. At this time, the taste values ​​(parameters) of the avatar held by the MPF and app are set. For example, different taste values ​​can be set for deformed, cute, polygon, etc. Note that parameters can be specified using the API.

[0132] It is necessary to read the necessary information and data from the binary data of the source data according to the conversion format. The format of the avatar to be converted has a data structure in the program. The format data is analyzed sequentially by a parser according to the data format that makes up the avatar data, such as materials, bones, meshes, animations, and coordinates. The analyzed data is then mapped to the structure of the destination data format and the conversion is performed.

[0133] Next, we will explain the taste processing procedure for switching between the two tastes mentioned above using Figure 27. The taste of the avatar is determined by processing it with mesh adjustment, which increases or decreases the number of meshes that form the basis of the 3D data. Taste processing when converting an avatar to match the MPF is an important process for optimizing appearance and expression for MPFs with different specifications.

[0134] If the mesh is increased, the avatar will be made rounder. If the mesh is decreased, the avatar will be made angular. The mesh is processed based on three-point mesh data. To convert a rounded model (data) to an angular model (data) (in the direction of arrow L1), follow the four steps below. In Figure 27, 2701, 2702, 2703, 2704, and 2705 indicate meshes.

[0135] (1) Starting from a certain mesh (for example, mesh 2701), vertices P1, P2, and P3 of adjacent meshes 2702, 2703, and 2704 are extracted. (2) The coordinates of each of the extracted vertices P1, P2, and P3 are stored. (3) A mesh 2705 (data) having three vertices P1, P2, and P3 is created. (4) Delete the data for meshes 2701, 2702, 2703, and 2704, and leave only the data for mesh 2705.

[0136] Conversely, to convert an angular model (data) into a rounded model (data) (in the direction of arrow L2), follow the five steps below. (1) Starting from a certain mesh (for example, mesh 2705), calculate the center of the edge so as to increase the number of vertices. For example, for one edge of mesh 2705, find the point (vertex P4) that is half the distance (center) between vertices P1 and P3. (2) Calculate the other two sides as above to find the center point of each. (3) Data for the points calculated in (2) above, that is, a triangular mesh 2706 having vertices P4, P5, and P6 (coordinates) is created. (4) Create data for meshes 2707, 2708, and 2709, with the vertices P4, P5, and P6 found in (3) above as vertices. (5) Delete the data for mesh 2705.

[0137] When the above taste processing procedure is executed as a program process, the taste of the avatar desired by the user is acquired, for example, as shown in FIG. 28. From the relationship shown in FIG. 26B, if the user selects the angular taste, the parameter "2" is set in advance. Therefore, the taste is acquired by acquiring the parameter (step S2801). Therefore, if the setting of the parameter "2" is confirmed, the taste is determined to be angular (step S2802). In this case, as will be described in detail in FIG. 29, mesh data reduction processing is executed to generate an angular model of the avatar (step S2803).

[0138] On the other hand, if the parameter is "1" in the taste acquisition, it is determined that the taste is rounded (step S2802), and as will be explained in detail in Fig. 30, a mesh data reduction process is executed to generate a rounded model of the avatar (step S2803). In this way, the taste of the avatar is determined.

[0139] Next, step S2803 mentioned above will be described in detail using Figure 29. To generate an angular avatar model, it is necessary to reduce the mesh data. Therefore, first, avatar mesh data is obtained (step S2901), and then, as shown in the image surrounded by a dashed line in the same figure, X, Y, and Z coordinates that define the arrangement of mesh 291 corresponding to that mesh data are obtained (step S2902).

[0140] Then, a search is performed for adjacent meshes using the two vertices of mesh 291, i.e., the X and Y coordinates, and as is clear from the image surrounded by a dashed line in the same figure, mesh 292 having X1Y1, Y and Z coordinates is detected (step S2903).

[0141] Similarly, a search is performed for adjacent meshes with two vertices of mesh 291, i.e., Y and Z coordinates, and mesh 293 with X2Z2, Y and Z coordinates is detected, as can be seen from the image surrounded by dashed lines in the same figure (step S2904).

[0142] Furthermore, a search is performed for adjacent meshes with two vertices of mesh 291, i.e., Z and X coordinates, and as is clear from the image surrounded by a dashed line in the same figure, mesh 294 with X2Z1, Y and Z coordinates is detected (step S2905).

[0143] Then, as shown by the image surrounded by a dashed line in the same figure, mesh data of an enlarged mesh structure having X1, Y1 coordinates of mesh 291, Y2, Z2 coordinates of mesh 293, and X2, Z1 coordinates of mesh 294 is created and stored as data after taste processing (step S2906).

[0144] Then, the mesh data of meshes 291 to 294 is deleted (step S2907), and the image enclosed by the dashed line in the figure is the mesh 295 after taste processing. In this way, mesh data reduction is executed.

[0145] Next, step S2804 described above will be described in detail using Figure 30. To generate a rounded avatar model, mesh data must be added. Therefore, first, avatar mesh data is obtained (step S3001), and then, as shown in the image surrounded by a dashed line in the same figure, X, Y, and Z coordinates that define the placement of mesh 391 corresponding to that mesh data are obtained (step S3002).

[0146] Then, as shown by the image surrounded by dashed lines in the figure, coordinates defining the midpoint between each two X, Y, and Z coordinates that define the vertices of mesh 391 are calculated (step S3003). The coordinates of the three midpoints calculated in step S3003 are defined as XY, XZ, and YZ, respectively, and mesh 392, which is 1 / 4 scale of mesh 391, is created (step S3004), as shown by the image surrounded by dashed lines in the figure.

[0147] Based on the XY, XZ, YZ coordinates of mesh 392 created in step S3004 and the three X, Y, Z coordinates of mesh 391, three meshes 393, 394, and 395 adjacent to mesh 392 are created, and the respective mesh data is stored (step S3005).

[0148] Then, the mesh data of the original mesh 391 surrounded by the dashed line WL in the figure is deleted (step S2907), and the image becomes four meshes 392 to 305 after taste processing. In this way, the mesh data is added.

[0149] Next, the reading of avatar data will be described with reference to Fig. 31. Fig. 31 is a flowchart illustrating the reading of avatar data according to this embodiment. In the MPF, first, when access is accepted from any of the HMD 10, PC, and mobile devices (steps 3101, 3102, 3103), user authentication is performed (step 3104).

[0150] If user authentication is successful, the MPF to be used is selected by user operation (step 3105), and the application / service is selected by user operation (step 3106). Furthermore, an avatar is selected by user operation. If a standard avatar provided by the application / service is selected (step 3107), avatar data is acquired (step 3108), and the application / service is executed using the standard avatar (step 3112).

[0151] Furthermore, when a common avatar to be created by the thin client is selected (step 3107), the user management unit accesses the user information to obtain avatar data of the common avatar (step 3110), and registers the avatar data (step 3111).Then, the application / service is executed using the common avatar (step 3112).

[0152] Next, user file operations will be described using Fig. 32. Fig. 32 is a flowchart illustrating user file operations according to this embodiment. In the MPF, first, when access is accepted from any of the HMD 10, PC, and mobile devices (steps S3201, S3202, S3203), user authentication is performed (step S3204). Because it is a thin client, authentication is performed automatically when the server is accessed.

[0153] If the user authentication is successful, the user selects the MPF to be used (step S3205), and sends an application launch request to the MPF (step S3206). In this way, the application / service is executed (step S3207).

[0154] The running application / service accesses the user management server and executes file access (step S3208). Next, the user management executes a user file operation (step S3209), and the application displays the operation status of the accessed file on the display (step S3210).

[0155] As described above, according to this embodiment, it is possible to use the metaverse and XR apps anytime and anywhere, regardless of the HMD or location. In particular, since an avatar only needs to be created once, and there is no need to create an avatar for each platform or app used, it is possible to easily prepare an avatar that can be used in any environment without worrying about the avatar's style. Furthermore, in a VR environment, user-owned content (images, videos) is stored on the platform, so it can be used interchangeably across platforms.

[0156] In particular, the reduction in the size of the HMD itself leads to a lighter weight, which allows for longer use and lowers the cost of the equipment, promoting its use in corporate activities. Furthermore, since the user environment is managed by the platform, it is possible to prevent fraudulent use by users who impersonate others.

[0157] It also allows users to access platforms from different vendors, and allows them to use their own content on any device, making content creation and data sharing more convenient.

[0158] It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications. Furthermore, the above-described embodiment has been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations.

[0159] Furthermore, some of the configurations of the above-described embodiments can be added to, deleted from, or replaced with other configurations. Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. Furthermore, the above-described configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function.

[0160] Information such as programs, tables, and files that realize each function may be stored in a memory, a recording device such as a hard disk or SSD, or a recording medium such as an IC card, SD card, or DVD. [Explanation of symbols]

[0161] 10 HMD 30 TCP 31 Thin Client Server 32 Communication equipment 33 Avatar Processing Unit 40 Thin Client Authentication Server 50 User Management Server 70 MPF 101 Camera 102 Display 103 Communication equipment 104 Input Device 105 CPU 106 memory 1067 Application Execution Environment 108 Middleware 109 Storage device 110 OS 121 User authentication processing unit 122 App Management Department 123 Communications Department 301 Avatar Creation Department 302 Avatar Conversion Department 401 Thin Client Authentication Unit 401,511,711,811 Input devices 402,512,712,812 output devices 403,513,713,813 CPU 404,514,714,814 memory 405,515,715,815 Communication equipment 406,516,716,816 Middleware 407,517,717,817 Storage device 407A, 517A, 717A, 817A User Information DB 408,518,718,818 OS 460,821 User authentication processing section 461 Biometric authentication processing unit 462 Multi-factor authentication processing section 463,823 User Management Department 464,724,824 Communications Department 470-1~470-N,570-1~570-N,770-1~770-N,870-1~870-N User Information 471,771,871 Personal Information 472 Biometric Information 473 Multi-Factor Authentication Information 474 Registration Platform Information 501 User Management Department 572 User Data 701 MPF authentication server 702 MPF Authentication Department 721 Authentication processing section 722 User Management Department 723 App Management Department 772 Credentials 773 Registered App / Service Information

Claims

1. A metaverse system having one or more metaverse platforms that provide a metaverse space for a head-mounted display connected to a network, a storage unit that stores user information for user authentication for each user who uses the metaverse space; an authentication unit that performs multi-stage user authentication on a user who uses a metaverse space through access from the head-mounted display; a provision unit that provides a usage environment of a metaverse space to the head-mounted display when all of the multi-stage user authentications are successful in the authentication unit; A metaverse system comprising:

2. In the metaverse system described in claim 1, the authentication unit performs multi-stage user authentication by performing first user authentication for the head-mounted display and second user authentication for the metaverse platform to be used.

3. In the metaverse system described in claim 2, each metaverse platform is characterized in having a platform-side memory unit that stores user information for user authentication for each user who uses the metaverse space for the second user authentication, and a platform-side authentication unit that performs second user authentication based on the user information stored in the platform-side memory unit.

4. 3. The metaverse system according to claim 2, wherein the first user authentication uses biometric information of the user received from the head-mounted display.

5. 2. The metaverse system according to claim 1, wherein the head-mounted display has a user authentication processing unit that cooperates with the authentication unit to perform the multi-stage user authentication.

6. The metaverse system of claim 1, further comprising an avatar creation unit that creates a single common avatar to be used in common on each of the metaverse platforms, and a user management unit that stores avatar data of the common avatar created by the avatar creation unit.

7. 7. The metaverse system according to claim 6, further comprising an avatar conversion unit that converts avatar data stored in said user management unit into a format of each of said metaverse platforms.

8. 8. The metaverse system according to claim 7, wherein the avatar data has a mesh structure, and the avatar conversion unit adjusts the mesh structure of the avatar data to process the taste.

9. The metaverse system of claim 1 further comprises one or more vendor servers that provide apps or services to the one or more metaverse platforms via the network, and the authentication unit further performs third user authentication for the apps or services provided on the metaverse platform to be used.

10. In the metaverse system described in claim 9, each vendor server is characterized in having a vendor-side memory unit that stores user information for user authentication for each user who uses the metaverse space for the third user authentication, and a vendor-side authentication unit that performs third user authentication based on the user information stored in the vendor-side memory unit.

11. A metaverse provision method of one or more metaverse platforms that provides a metaverse space to a head-mounted display connected to a network, comprising: A metaverse provision method characterized by storing user information for user authentication in memory for each user using the metaverse space, performing multi-stage user authentication on the user using the metaverse space in response to access from the head-mounted display, and providing a usage environment for the metaverse space to the head-mounted display if all of the multi-stage user authentications are successful.

12. The metaverse provision method according to claim 11, characterized in that multi-stage user authentication is performed by first user authentication for the head-mounted display and second user authentication for the metaverse platform to be used.

13. The metaverse provision method of claim 12, further comprising connecting one or more vendor servers that provide apps or services to the one or more metaverse platforms via the network, and further comprising performing third user authentication for the apps or services provided on the metaverse platform to be used.

Images