System

Abstract

To provide a system allowing a user to smoothly collect information and progress an event.SOLUTION: A system for providing a Metaverse and allowing a user to play a game includes means for a server to authenticate a user and generate a session ID, means for rendering a Metaverse based on scene data and character data, means for displaying a purpose of a game, means for allowing a user to ask a question to a character in the Metaverse and displaying a response to the question to the user, means for generating event information based on action information of the user and displaying the event information to the user, and means for allowing the server to verify achievement of the purpose of the user and generate and display reward information.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional metaverse systems have problems with the smooth progress of information gathering and events required for users to achieve their goals. Furthermore, when users cooperate to achieve a goal, they may not be able to properly interact with in-game characters, which can detract from the gaming experience. To solve these problems, a system that allows users to smoothly gather information and progress through events is needed. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a system including: means for transmitting authentication information for a user to log in to the metaverse; means for a server to receive the authentication information and authenticate the user; means for the server to generate a session ID for the authenticated user; means for a terminal to request scene data and character data from the server; means for the server to acquire the scene data and character data and send it to the terminal; means for the terminal to render the metaverse based on the scene data and character data received; means for the terminal to request a game objective from the server and display the received objective; means for the user to send a question to a character in the metaverse; means for the server to generate a response to the question and send it to the terminal; means for the terminal to display the response received by the user; means for sending user behavior information to the server and confirming the event occurrence conditions; means for the server to generate event information and send it to the terminal if the conditions are met; means for the server to display the event information received by the terminal to the user; means for the server to verify that the user has achieved their objective and generate reward information; and means for sending the reward information to the terminal and displaying it.

[0006] A "user" is a person or entity that accesses the Metaverse system and plays games.

[0007] The "metaverse" is an interactive three-dimensional environment provided in virtual space, a digital world where users can log in and interact.

[0008] "Logging in" is the process by which a user provides authentication information to access a system and is then authorized to access the system.

[0009] "Authentication information" is data (such as username and password) that a user provides to identify themselves to a system.

[0010] A "server" is a computer system that receives requests from terminals via a network and performs various processes such as authentication, data processing, and event management.

[0011] "Authentication" is the process of verifying that the authentication information provided is valid and granting the user access.

[0012] A "session ID" is a unique identifier assigned by a server to an authenticated user, used to track and manage the user's activities.

[0013] "Scene data" is data that includes information about the terrain, buildings, objects, etc. of the Metaverse environment.

[0014] "Character data" refers to data that includes information such as the profile, abilities, and location of characters that exist within the metaverse.

[0015] "Rendering" is the process of generating a visually displayable 3D environment based on scene data and character data.

[0016] The "objective of the game" is the content of the tasks or missions that the user must accomplish within the metaverse.

[0017] A "question" is a word or request that a user poses to a character in order to obtain information necessary to progress in the game.

[0018] A "character" is a virtual entity (such as an NPC) that interacts with the user and other characters within the metaverse.

[0019] A "response" is information or an action given by a character in response to a question.

[0020] "Behavioral information" is data about a series of actions or behaviors a user takes within the metaverse.

[0021] An "Event" is an occurrence or scenario that occurs within the Metaverse under certain conditions.

[0022] "Reward information" is data relating to the reward given to a user when the user achieves the objective of the game. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] Basic system overview

[0045] The present invention is a system in which a user logs into the Metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. The operation of the system will be described in detail below for each processing step.

[0046] Overall system configuration

[0047] The system consists of the following main components:

[0048] 1. User Device

[0049] 2. Server System

[0050] 3. Metaverse Database

[0051] Login process

[0052] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[0053] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0054] Loading the Metaverse

[0055] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[0056] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[0057] Presentation of the game objective

[0058] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[0059] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0060] Cooperation and information gathering

[0061] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0062] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0063] Event Progression

[0064] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[0065] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[0066] Achieving goals and distributing rewards

[0067] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[0068] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[0069] The above is a specific description of the embodiment for carrying out the present invention.

[0070] The processing flow will be explained below.

[0071] Program processing steps

[0072] Step 1:

[0073] User Login

[0074] The user opens the login form using the terminal.

[0075] The terminal presents the user with a form for entering a username and password.

[0076] The user enters their authentication information and clicks the Login button.

[0077] The device sends the authentication information to the server.

[0078] The server checks the authentication information against a database and determines whether the authentication is successful.

[0079] If the server is successful in the authentication, it generates a session ID and sends it to the terminal.

[0080] The terminal displays a successful login message.

[0081] Step 2:

[0082] Loading the Metaverse

[0083] The terminal sends a request for scene data and character data to the server.

[0084] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[0085] The server acquires the character data and sends it to the device.

[0086] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[0087] Step 3:

[0088] Presentation of the game objective

[0089] The terminal requests game objective information from the server.

[0090] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[0091] The terminal displays the acquired game objective to the user.

[0092] Step 4:

[0093] Cooperation and information gathering

[0094] Users approach and interact with characters in the Metaverse.

[0095] The user inputs a question and sends it to the character.

[0096] The terminal transmits the query information to the server.

[0097] The server uses the character's AI engine to generate a response and send it to the device.

[0098] The terminal displays the response to the user.

[0099] Step 5:

[0100] Event Progression

[0101] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[0102] The server receives the user's behavior information and checks the event occurrence conditions.

[0103] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[0104] The device displays event information to the user and prompts them to take the next action.

[0105] Step 6:

[0106] Achieving goals and distributing rewards

[0107] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[0108] The server verifies the achievement of the objectives and generates reward information.

[0109] The server transmits the reward information to the terminal.

[0110] The terminal displays reward information to the user and notifies them that the game session is ending.

[0111] Specific example explanation

[0112] As a concrete example, we will explain the specific actions that "User A" takes from logging in to achieving his or her goal.

[0113] Step 1:

[0114] "User A" uses a terminal to open the login form page, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which checks the authentication information in the database. If the authentication is successful, the server generates a session ID and sends it back to the terminal. "User A"'s terminal displays a login success message.

[0115] Step 2:

[0116] After "User A" successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database and sends it to "User A's" device. The device renders the metaverse based on the received data and displays it to "User A."

[0117] Step 3:

[0118] "User A" operates the device to confirm the objective of the game, and the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays the information to "User A" and presents the objective of "find the lost treasure."

[0119] Step 4:

[0120] "User A" approaches a villager character (CP) and asks "Where is the treasure?" The terminal of "User A" sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the terminal. The terminal displays the response to "User A."

[0121] Step 5:

[0122] When "User A" heads towards the castle, the device sends this action information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates event information that "the castle gates have been magically closed" and sends it to the device. The device then displays this event information to "User A" and prompts him to take the next action.

[0123] Step 6:

[0124] "User A" collects the necessary items and finds the treasure. The device sends the information to the server, which verifies that the objective was achieved. The server generates reward information (e.g., in-game currency) and sends it to the device. The device displays the reward information to "User A" and notifies him or her that the game session is ending.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] Current virtual reality systems require complex system processing after a user logs in, which can degrade the quality of the user experience. Furthermore, when users interact with characters in the virtual space, conventional systems only provide limited responses, resulting in a lack of completeness as a comprehensive gaming environment. Furthermore, there are many instances where manual processing is required for reward distribution when a user achieves a goal, and automation is needed.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes means for receiving and verifying authentication information for a user to log in to a virtual space, means for generating a session ID for the authenticated user, and means for generating question and answer prompts for a character in the virtual space using a generative AI model. This simplifies system processing after a user logs in, improving the quality of the user experience. It also enables more diverse and realistic responses in character interactions in the virtual space, improving the overall game environment. Furthermore, it automates reward allocation upon goal achievement, increasing overall efficiency.

[0130] "Authentication information" refers to a combination of a username and password that a user uses to log in to a virtual space.

[0131] A "server" is a central computer system for the entire virtual space system, providing data storage, processing, user authentication, and other functions.

[0132] A "session ID" is a unique identifier generated by the server to identify a user after the user has been authenticated.

[0133] A "terminal" is a device used by a user to access a virtual space, and includes a PC, smartphone, tablet, etc.

[0134] "Scene data" is data that constitutes the environment, background, objects, etc. within a virtual space.

[0135] "Character data" refers to data that includes information, attributes, actions, etc. of characters in a virtual space.

[0136] "Rendering" is the process of visually displaying a virtual 3D environment based on scene data and character data.

[0137] The "objective of the game" is the goal or task that the user must achieve within the virtual space.

[0138] A "generative AI model" is an algorithm or model that uses artificial intelligence technology to generate appropriate responses or content based on user input.

[0139] "Behavioral information" is data on the actions and choices made by the user within the virtual space.

[0140] An "event occurrence condition" is a condition or trigger element necessary for a specific event to occur in virtual space.

[0141] "Event information" is information that is generated when an event occurrence condition is met, and includes details of the event and the next action to be taken.

[0142] "Reward information" is details of the rewards a user will receive when they achieve a goal in the game. Examples include virtual currency, items, etc.

[0143] The present invention is a system in which a user logs into a virtual space and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Each component and its operation will be described in detail below.

[0144] User terminal

[0145] User terminals are devices such as PCs, smartphones, and tablets. Users use these terminals to access the virtual space and perform everything from login to game progression. The terminals use software such as HTML, JavaScript (registered trademark), React, and Three.js to execute the user interface and 3D rendering.

[0146] Server System

[0147] The server system is a central computer system that handles user authentication, database management, response generation using generative AI models, event management, etc. The server runs on software such as Node.js and MySQL (registered trademark).

[0148] Metaverse Database

[0149] The metaverse database stores user information, scene data, character data, game objectives, event information, etc. MySQL is used as the database management system.

[0150] Login process

[0151] The user displays a login form on their device and enters their username and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a new session ID and returns it to the device. The device notifies the user of successful authentication and displays an entry point to the virtual space.

[0152] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0153] Loading the Metaverse

[0154] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders a 3D virtual environment based on the received data and displays it to the user.

[0155] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the virtual space based on that data.

[0156] Presentation of the game objective

[0157] The device requests game objective information from the server. The server retrieves the objective information from a database based on the user's session ID. The server sends the objective information to the device, which displays it to the user.

[0158] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0159] Cooperation and information gathering

[0160] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine (generative AI model) to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0161] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0162] Prompt Sentence Examples

[0163] User prompt: "Where is the treasure?"

[0164] Event Progression

[0165] When a user takes a specific action (e.g., heads to the castle), the device sends the action information to the server. The server checks the event occurrence conditions and generates event information if the conditions are met. The server then sends the event information to the device, which then displays it to the user.

[0166] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[0167] Achieving goals and distributing rewards

[0168] When the user achieves the game objective, the device sends that information to the server. The server confirms the achievement and generates reward information (e.g., in-game currency). The server then sends the reward information to the device, which displays it to the user. This ends the game session.

[0169] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information, and sends it to the device. The device displays it to "User A" and notifies him that the game is over.

[0170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0171] Step 1:

[0172] The user enters their username and password into the login form on their device.

[0173] Input: Username, Password

[0174] Specific behavior: The device displays the login screen using HTML and JavaScript. The user fills out the form and clicks the login button.

[0175] Step 2:

[0176] The terminal transmits the entered authentication information to the server.

[0177] Input: Username, Password

[0178] Specific operation: Sends authentication information to the server via an HTTP POST request using the JavaScript Fetch API.

[0179] Step 3:

[0180] The server checks the credentials against a database.

[0181] Input: Username, Password

[0182] Output: Authentication success or failure, session ID

[0183] What happens: The server uses Node.js to query the MySQL database for authentication information. If the credentials match, it generates a new session ID with a UUID.

[0184] Step 4:

[0185] The server returns the authentication result to the terminal and includes the new session ID.

[0186] Input: Authentication success or failure, session ID

[0187] Output: Session ID

[0188] Specific operation: If authentication is successful, the session ID is returned to the terminal in the HTTP response.

[0189] Step 5:

[0190] The terminal notifies the user of successful authentication and displays an entry point to the virtual space.

[0191] Input: Session ID

[0192] Specific behavior: The device displays a successful authentication message and an entry point link. The screen is updated using React.

[0193] Step 6:

[0194] The user clicks on an entry point into the virtual space.

[0195] Input: Entry point URL, Session ID

[0196] Specific operation: The terminal sends a request to the entry point to the server.

[0197] Step 7:

[0198] The terminal sends a request for scene data and character data to the server.

[0199] Input: Session ID

[0200] Specific operation: The device requests scene data and character data from the server using an HTTP GET request.

[0201] Step 8:

[0202] The server retrieves scene data and character data from the database and transmits them to the terminal.

[0203] Input: Session ID

[0204] Output: Scene data, character data

[0205] Specific operation: The server uses Node.js to retrieve the necessary data from the MySQL database and send it to the terminal as an HTTP response.

[0206] Step 9:

[0207] The device renders a 3D virtual environment based on the scene data and character data it receives and displays it to the user.

[0208] Input: Scene data, character data

[0209] What it does: The device uses Three.js to render the scene and characters in 3D and display them in the user interface.

[0210] Step 10:

[0211] The terminal requests information for game purposes from the server.

[0212] Input: Session ID

[0213] Specific operation: The device requests the game objective from the server via an HTTP GET request.

[0214] Step 11:

[0215] The server retrieves game objective information from the database based on the user's session ID and sends it to the terminal.

[0216] Input: Session ID

[0217] Output: Game objective information

[0218] Specific operation: The server uses Node.js to retrieve game objective information from the MySQL database and sends it to the terminal as an HTTP response.

[0219] Step 12:

[0220] The terminal displays the received game objective information to the user.

[0221] Input: Game objective information

[0222] Specific operation: The device uses React to display game objective information on the user interface.

[0223] Step 13:

[0224] The user sends a question to a character in the virtual space.

[0225] Input: User's question

[0226] Specific operation: The device sends the question to the server.

[0227] Step 14:

[0228] The terminal sends the user's question to the server.

[0229] Input: User's question

[0230] Specific operation: The question is sent to the server via an HTTP POST request, possibly using WebSocket.

[0231] Step 15:

[0232] The server uses an AI engine (generative AI model) to generate a response to the question and send it to the terminal.

[0233] Input: User's question

[0234] Output: Response by AI engine

[0235] Specific operation: The server uses a generative AI model (e.g., GPT-3 (registered trademark)) to generate an answer to the question and sends it to the terminal in an HTTP response.

[0236] Step 16:

[0237] The terminal displays the received response to the user.

[0238] Input: Response by AI engine

[0239] Specific behavior: The device displays the response in the user interface using React.

[0240] Step 17:

[0241] The user takes a specific action (e.g., heads to the castle).

[0242] Input: User behavior information

[0243] Specific operation: The device sends its behavior information to the server.

[0244] Step 18:

[0245] The terminal transmits the user's behavior information to the server.

[0246] Input: User behavior information

[0247] Specific operation: Sends behavioral information to the server via an HTTP POST request.

[0248] Step 19:

[0249] The server checks the event occurrence conditions, and if the conditions are met, generates event information and sends it to the terminal.

[0250] Input: User behavior information

[0251] Output: Event information

[0252] Specific operation: The server checks the event occurrence conditions based on the behavioral information, and if the conditions are met, it generates event information using Node.js and sends it to the terminal as an HTTP response.

[0253] Step 20:

[0254] The terminal displays the received event information to the user.

[0255] Input: Event information

[0256] Specific behavior: The device displays the event information on the user interface using React.

[0257] Step 21:

[0258] The user completes the game objective (e.g., finds treasure).

[0259] Input: Objective achievement information

[0260] Specific operation: The device sends the information to the server.

[0261] Step 22:

[0262] The terminal transmits the goal achievement information to the server.

[0263] Input: Objective achievement information

[0264] Specific behavior: Sends achievement information to the server using an HTTP POST request.

[0265] Step 23:

[0266] The server verifies the achievement of the objectives and generates reward information.

[0267] Input: Objective achievement information

[0268] Output: Reward information

[0269] Specific operation: The server confirms that the goal has been achieved and generates reward information using Node.js.

[0270] Step 24:

[0271] The server transmits the reward information to the terminal.

[0272] Input: Reward information

[0273] Specific operation: Reward information is sent to the terminal in an HTTP response.

[0274] Step 25:

[0275] The terminal displays the received reward information to the user.

[0276] Input: Reward information

[0277] Specific behavior: The device displays the reward information on the user interface using React and notifies the end of the game session.

[0278] (Application example 1)

[0279] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] There is a need for an effective method to facilitate interactions with characters and event progression in order for users to achieve specific goals in the metaverse. There is also a need to make the shopping experience more attractive by allowing users to receive special benefits such as coupons when purchasing specific products in virtual stores.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0282] In this invention, the server includes: means for sending authentication information for a user to log in to the metaverse; means for the server to receive the authentication information and authenticate the user; means for the server to generate a session ID for the authenticated user; means for the terminal to request scene data and character data from the server; means for the server to acquire the scene data and character data and send it to the terminal; means for the terminal to render the metaverse based on the scene data and character data received; means for the terminal to request a game objective from the server and display the received objective; means for the user to send a question to a character in the metaverse; means for the server to generate a response to the question and send it to the terminal; means for the terminal to display the response received by the user; means for sending user behavior information to the server and confirming the event occurrence conditions; means for the server to generate event information and send it to the terminal if the conditions are met; means for the server to display the event information received by the terminal to the user; means for the server to verify the user's achievement of the objective and generate reward information; means for sending the reward information to the terminal and displaying it; means for generating a coupon when the user purchases a specific product and sending it to the terminal; and means for receiving a prompt message regarding the purchase of a specific product using a generative AI model. This allows users to smoothly interact with and progress through events within the metaverse, and also makes it possible to offer coupons when purchasing certain products.

[0283] The "metaverse" is a three-dimensional virtual environment in which users can interact with other users and virtual characters within a virtual space.

[0284] "Authentication information" refers to information such as a username and password that a user uses when accessing the metaverse.

[0285] A "session ID" is a unique identification number generated by the server for an authenticated user.

[0286] "Scene data" is data about the environment and objects within the metaverse.

[0287] "Character data" refers to data about characters and NPCs (Non-Player Characters) controlled by users within the Metaverse.

[0288] "Rendering" is the process by which a computer generates a visually displayable image from three-dimensional scene data.

[0289] A "coupon" is electronic information indicating a discount or benefit that is offered when a user meets certain conditions.

[0290] A "prompt sentence" is textual input information that a generative AI model uses to generate a response.

[0291] A "generative AI model" is an artificial intelligence model that generates an appropriate response based on a given prompt sentence.

[0292] "Event information" is information about specific actions or events that occur within the metaverse in response to user behavior conditions.

[0293] "Reward information" is information regarding rewards that will be provided to users when they achieve specific goals within the metaverse.

[0294] A "virtual store" is a store that exists virtually on the Internet, and is a virtual space where users can gather product information and make purchases.

[0295] Basic system overview

[0296] The present invention is a system that allows users to log in to a virtual store and collect product information to achieve specific goals, or to receive rewards such as coupons as they progress through events. The operation of the system is described in detail below.

[0297] Overall system configuration

[0298] The system mainly consists of the following major components:

[0299] 1. User device: Smartphone application, etc.

[0300] 2. Server system: A server that processes and manages data.

[0301] 3. Database: A database that manages metaverse scene data, character data, product data, etc.

[0302] 4. Generative AI model: An artificial intelligence model that generates appropriate responses or coupons based on user prompts.

[0303] Login process

[0304] The user accesses the virtual store using their terminal. A login form is displayed, the user enters their authentication information (username and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database. If authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authorized and the entry point to the virtual store is displayed.

[0305] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends "User A's" authentication information to the server, which then verifies it in the database, generates a session ID, and sends it to "User A's" terminal.

[0306] Loading a Virtual Store

[0307] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the terminal. The terminal then renders a 3D environment of the virtual store based on this data and displays it to the user.

[0308] As a concrete example, the terminal of "User A" requests scene data and product data from the server, and the server retrieves this data from the database and sends it to the terminal of "User A." The terminal then displays a virtual store based on that data.

[0309] State your shopping purpose

[0310] The terminal requests shopping purpose information from the server, and the server obtains the purpose information based on the user's session ID and sends it to the terminal, which then displays it to the user, allowing them to take the first step toward achieving their purpose.

[0311] For example, "User A" sends a request to the server to confirm his / her shopping purpose on his / her device, and the server acquires the purpose information "to find discount sales" and sends it to "User A's" device. The device displays it, allowing "User A" to recognize his / her purpose.

[0312] Cooperation and information gathering

[0313] Users can interact with store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question, the device sends the question to the server. The server uses a generative AI model to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0314] For example, "User A" asks a store clerk character for "details of the most popular item." The device sends the question to the server, and the server generates a response saying, "This item is currently on sale, 20% off," and sends it to the device. The device displays it to "User A."

[0315] Event Progression

[0316] When a user purchases a specific product, the behavioral information is sent from the device to the server. The server checks the purchase conditions, and if the conditions are met, the server generates event information and sends it to the device. The device displays the event information to the user, prompting them to take the next action.

[0317] For example, when "User A" places a specific product in their cart and purchases it, their device sends that information to the server. The server confirms that the conditions are met, generates event information for a "10% off coupon for your next purchase," and sends it to their device. The device then displays this to "User A."

[0318] Achieving goals and distributing rewards

[0319] When the user achieves their shopping goal, the terminal sends the information to the server. The server verifies the goal achievement and generates reward information and sends it to the terminal. The terminal displays the reward information to the user, and the shopping session ends.

[0320] As a concrete example, when "User A" achieves his / her goal in a virtual store, the device sends this information to the server. The server confirms that the goal has been achieved, generates a "coupon for next use" and sends it to the device. The device displays this to "User A" and notifies him / her that his / her shopping has finished.

[0321] An example prompt is:

[0322] "Generate a 10% off coupon for the next purchase by the user when they purchase a specific product."

[0323] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0324] Step 1:

[0325] A user accesses a virtual store from a terminal, enters authentication information (user name and password) in the login form, and presses the login button. The entered authentication information is sent from the terminal to the server. The server compares the received authentication information against a database and performs authentication. If authentication is successful, the server generates a new session ID and sends it back to the terminal. The terminal then notifies the user that access to the system has been permitted, and displays the entry point to the virtual store.

[0326] Input: Authentication information (username, password)

[0327] Output: Session ID

[0328] Step 2:

[0329] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves the metaverse scene data and product data from the database based on the session ID and sends them to the terminal. The terminal then renders the 3D environment of the virtual store based on this data and displays it to the user.

[0330] Input: Session ID

[0331] Output: Scene data and product data

[0332] Step 3:

[0333] The terminal sends a request for shopping intent information to the server. The server retrieves the intent information from the database based on the session ID and sends it to the terminal. The terminal displays the intent information to the user and prompts them to take their first action.

[0334] Input: Session ID

[0335] Output: Shopping intent information

[0336] Step 4:

[0337] Users can ask questions to store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question such as "Details of the most popular item," the device sends the question to the server. The server uses a generative AI model to generate an appropriate response to the question and sends it to the device. The device then displays the received response to the user.

[0338] Input: User prompt

[0339] Output: Response

[0340] Step 5:

[0341] When a user places a specific product in their cart and purchases it, that behavioral information is sent from the device to the server. The server checks the purchase conditions based on this information, and if the conditions are met, generates event information and sends it to the device. For example, if the conditions are met, the server generates a "10% off coupon for your next purchase" and sends it to the device. The device then displays this coupon information to the user.

[0342] Input: Purchase Information

[0343] Output: Coupon information

[0344] Step 6:

[0345] When the user achieves their shopping goal, the terminal sends the information to the server, which verifies the goal achievement, generates reward information, and sends it to the terminal, which displays the reward information to the user and notifies them that the shopping session has ended.

[0346] Input: Objective achievement information

[0347] Output: Reward information

[0348] The above are the specific processing steps of the system of the present invention. This allows users to smoothly interact with the virtual store and progress through events. Furthermore, coupons can be obtained when purchasing specific products, which is expected to improve the shopping experience.

[0349] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0350] Basic system overview

[0351] The present invention is a system in which a user logs into the metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience within the metaverse based on the user's emotions.

[0352] Overall system configuration

[0353] The system consists of the following main components:

[0354] 1. User Device

[0355] 2. Server System

[0356] 3. Metaverse Database

[0357] 4. Emotion Engine

[0358] Login process

[0359] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[0360] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0361] Loading the Metaverse

[0362] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[0363] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[0364] Presentation of the game objective

[0365] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[0366] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0367] Cooperation and information gathering

[0368] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0369] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0370] Event Progression

[0371] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[0372] As a concrete example, when "User A" heads to the castle, the device sends this behavioral information to the server. The server confirms that the conditions are met, generates event information indicating that the castle gate has been magically closed, and sends it to the device. The device then displays this event information to "User A."

[0373] Achieving goals and distributing rewards

[0374] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[0375] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[0376] Incorporating an emotion engine

[0377] The emotion engine analyzes the user's facial expressions and voice to recognize their emotions, which are then sent to the server, which then dynamically adjusts events and interactions within the metaverse.

[0378] As a concrete example, when "User A" talks to a villager character in the metaverse, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays it to "User A."

[0379] The introduction of the emotion engine personalizes the gaming experience based on the real-time emotions felt by the user, providing an even more immersive experience.

[0380] The processing flow will be explained below.

[0381] Program processing steps

[0382] Step 1:

[0383] User Login

[0384] The user opens the login form using the terminal.

[0385] The terminal presents the user with a form for entering a username and password.

[0386] The user enters their authentication information and clicks the Login button.

[0387] The device sends the authentication information to the server.

[0388] The server checks the authentication information against a database and determines whether the authentication is successful.

[0389] If the authentication is successful, the server generates a session ID and sends it to the terminal.

[0390] The terminal displays a successful login message.

[0391] Step 2:

[0392] Loading the Metaverse

[0393] The terminal sends a request for scene data and character data to the server.

[0394] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[0395] The server also obtains character data and sends it to the device.

[0396] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[0397] Step 3:

[0398] Early recognition of emotions

[0399] When a user logs into the metaverse on their device, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotions.

[0400] The emotion engine transmits the recognized emotion information to the server.

[0401] The server stores the emotional information in a database and uses it in subsequent gameplay.

[0402] Step 4:

[0403] Presentation of the game objective

[0404] The terminal requests game objective information from the server.

[0405] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[0406] The terminal displays the acquired game objective to the user.

[0407] Step 5:

[0408] Cooperation and information gathering

[0409] Users approach and interact with characters in the Metaverse.

[0410] The user inputs a question and sends it to the character.

[0411] The terminal transmits the query information to the server.

[0412] The server uses the character's AI engine to generate a response and send it to the device.

[0413] The terminal displays the response to the user.

[0414] Step 6:

[0415] Emotional remonitoring

[0416] The emotion engine periodically analyzes the user's facial expressions and voice, recognizing emotions in real time.

[0417] The recognized emotion information is sent back to the server.

[0418] The server dynamically adjusts events and character responses based on the emotional information.

[0419] Step 7:

[0420] Event Progression

[0421] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[0422] The server receives the user's behavior information and checks the event occurrence conditions.

[0423] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[0424] The device displays event information to the user and prompts them to take the next action.

[0425] Step 8:

[0426] Achieving goals and distributing rewards

[0427] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[0428] The server verifies the achievement of the objectives and generates reward information.

[0429] The server transmits the reward information to the terminal.

[0430] The terminal displays reward information to the user and notifies them that the game session is ending.

[0431] Examples of emotion engines

[0432] As a concrete example, we will explain the process by which "User A" logs in to the Metaverse and plays.

[0433] Step 1:

[0434] "User A" opens the login form on the terminal, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which verifies it against the database. If authentication is successful, the server sends a session ID, and the terminal displays a login success message.

[0435] Step 2:

[0436] After "User A" logs in, the device sends a request for scene and character data to the server. The server retrieves the data and sends it to the device. The device renders the metaverse and displays it to the user.

[0437] Step 3:

[0438] When "User A" enters the metaverse, the device's built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotions and transmits that information to the server, which stores the emotion information and uses it for subsequent processing.

[0439] Step 4:

[0440] In order for "User A" to confirm the objective of the game, the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays it to "User A" and presents the objective of "find the treasure."

[0441] Step 5:

[0442] "User A" asks a villager character "Where is the treasure?" The device sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the device. The device displays it to "User A."

[0443] Step 6:

[0444] While "User A" is interacting with a villager, the emotion engine recognizes tension from "User A's" facial expression. The server receives this information and adjusts the villager character's response to encourage relaxation. The device then converts the villager character's response into a suggestion for "what to do to relax" and displays it to the user.

[0445] Step 7:

[0446] When "User A" heads to the castle, the device sends that behavioral information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates new event information stating that "the castle gates have been magically closed" and sends it to the device. The device displays this to "User A" and prompts them to take the next action.

[0447] Step 8:

[0448] When "User A" finds the treasure, the device sends the achievement information to the server. The server confirms the achievement, generates reward information (e.g., in-game currency), and sends it to the device. The device displays the information to "User A" and notifies him or her that the game session is ending.

[0449] Example 2

[0450] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0451] Conventional virtual reality systems lack the ability to recognize users' emotions in real time and dynamically adjust the experience based on those emotions, which results in a lack of immersion and satisfaction for users.

[0452] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a session ID for an authenticated user, a means for acquiring scene data and character data and transmitting them to the terminal, and a means for transmitting user behavior information to the server and checking the event occurrence conditions. This allows the user's emotions to be recognized in real time after logging in, and the experience in the virtual space to be dynamically adjusted.

[0453] "Authentication information" is data such as a username and password that a user uses to access a system.

[0454] A "session ID" is a unique identifier generated by the server to identify operations by an authenticated user.

[0455] "Scene data" is data that includes information about the environment and objects in a virtual space.

[0456] "Character data" is data that includes information about characters in a virtual space.

[0457] A "virtual space" is a computer-generated 3D environment that can be accessed and experienced by a user.

[0458] "Rendering" is the process by which a computer generates a graphical display based on scene data and character data.

[0459] "Game objective" refers to the tasks or goals that a user must accomplish within the virtual space.

[0460] "Event information" is information about an event that occurs when certain conditions are met in the virtual space.

[0461] "Reward information" is information about the reward that a user can receive when they achieve the objective of the game.

[0462] An "emotion engine" is software that recognizes emotions in real time from a user's facial expressions, voice, etc.

[0463] A "virtual character" is a computer-generated character that interacts with a user in a virtual space.

[0464] "Interaction" refers to the actions or operations that a user performs to interact with elements in a virtual space.

[0465] Basic system overview

[0466] The present invention is a system in which a user logs into a virtual space and cooperates with other characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience in the virtual space based on the user's emotions.

[0467] Overall system configuration

[0468] The system consists of the following main components:

[0469] 1. User Device

[0470] 2. Server System

[0471] 3. Virtual Space Database

[0472] 4. Emotion Engine

[0473] Hardware and software used

[0474] User device: Computer or smartphone (e.g., Windows PC, smartphone)

[0475] Server systems: high-performance servers (e.g. cloud infrastructure)

[0476] Virtual spatial database: Data management system (e.g., MySQL, NoSQL database)

[0477] Emotion engine: Emotion recognition software (e.g., Emotion SDK)

[0478] Login process

[0479] A user accesses the virtual space system using their own terminal. A login form is displayed, and the user enters authentication information (user name and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authenticated, and the entry point to the virtual space is displayed. As a concrete example, "User A" attempts to log in on the terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0480] Loading the virtual world

[0481] When a user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders and displays a 3D environment in the virtual space based on this data. As a concrete example, after "User A" has successfully logged in, the device requests scene data and character information from the server. The server retrieves this data and sends it to "User A's" device, and the device then displays the virtual space based on that data.

[0482] Presentation of the game objective

[0483] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays this to the user, allowing them to take the first step towards achieving their objective. As a concrete example, when "User A" wants to confirm the game objective, the device sends a request to the server. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0484] Cooperation and information gathering

[0485] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device displays the response to the user. As a concrete example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0486] Event Progression

[0487] When a user takes a specific action (for example, heading towards the castle), the device sends that action information to the server, and the server checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device displays that event information to the user and prompts them to take the next action. As a specific example, when "User A" heads towards the castle, the device sends that action information to the server. The server checks that the conditions are met, generates event information that the castle gates have been magically closed, and sends it to the device. The device displays that event information to "User A."

[0488] Achieving goals and distributing rewards

[0489] When a user achieves the game objective, the device sends that information to the server. The server verifies that the objective has been achieved, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends. As a specific example, when "User A" finds treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays it to "User A," notifying him that the game is over.

[0490] Incorporating an emotion engine

[0491] The emotion engine recognizes emotions by analyzing the user's facial expressions and voice. The recognized emotional information is sent to the server, which then dynamically adjusts events and interactions in the virtual space based on that information. As a specific example, when "User A" talks to a villager character in the virtual space, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays this to "User A." The introduction of the emotion engine personalizes the game experience based on the user's real-time emotions, providing an even more immersive experience.

[0492] Prompt Sentence Examples

[0493] "User A logs into a virtual world and works with his friends to find treasure. On the way to the castle, the emotion engine recognizes the user's stress, and a villager character suggests ways for the user to relax. Please explain this scenario in natural language."

[0494] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0495] Step 1: Login process

[0496] 1. Input: Authentication information (user name, password)

[0497] 2. Specific operation:

[0498] The user enters their username and password into the login form.

[0499] The terminal transmits the entered authentication information to the server.

[0500] 3. Data processing and calculation:

[0501] The server checks the received authentication information against its database.

[0502] A database query is performed to verify that the username and password match.

[0503] 4. Output: Authentication result, session ID (if successful)

[0504] 5. Specific operation:

[0505] If the authentication is successful, the server generates a new session ID and sends it to the terminal.

[0506] The terminal receives the session ID and notifies the user that the login was successful.

[0507] Step 2: Loading the Virtual Space

[0508] 1. Input: Session ID

[0509] 2. Specific operation:

[0510] After the user has successfully logged in, the terminal sends a request for scene data and character data to the server.

[0511] 3. Data processing and calculation:

[0512] The server retrieves scene data and character data from the database based on the session ID.

[0513] Execute database queries and extract the required data.

[0514] 4. Output: Scene data, character data

[0515] 5. Specific operation:

[0516] The server transmits the acquired data to the terminal.

[0517] The device renders a 3D environment based on the data it receives and displays the virtual space.

[0518] Step 3: Present the objective of the game

[0519] 1. Input: Session ID

[0520] 2. Specific operation:

[0521] The terminal requests game objective information from the server.

[0522] 3. Data processing and calculation:

[0523] The server retrieves the desired information from the database based on the session ID.

[0524] Execute a database query to extract the desired information.

[0525] 4. Output: Game objective information

[0526] 5. Specific operation:

[0527] The server transmits the target information to the terminal.

[0528] The terminal displays the received objective information to the user and guides the user to the next step.

[0529] Step 4: Collaborate and gather information

[0530] 1. Input: User's question

[0531] 2. Specific operation:

[0532] The user inputs a question to the character and sends it.

[0533] The device sends the question to the server.

[0534] 3. Data processing and calculation:

[0535] The server uses a generative AI model to generate answers to questions.

[0536] Feed the AI ​​model with questions and generate appropriate responses.

[0537] 4. Output: Response from the character

[0538] 5. Specific operation:

[0539] The server generates a response and sends it to the terminal.

[0540] The terminal displays the response to the user.

[0541] Step 5: Event Progression

[0542] 1. Input: User behavior information

[0543] 2. Specific operation:

[0544] The user takes a specific action (e.g., heads to the castle).

[0545] The terminal transmits the behavior information to the server.

[0546] 3. Data processing and calculation:

[0547] The server executes a database query to check the conditions under which the event occurs.

[0548] If the condition is met, new event information is generated.

[0549] 4. Output: Event information

[0550] 5. Specific operation:

[0551] The server transmits the event information to the terminal.

[0552] The device displays the event information to the user and guides them on the next action.

[0553] Step 6: Achieving Objectives and Distributing Rewards

[0554] 1. Input: Objective achievement information

[0555] 2. Specific operation:

[0556] When the user achieves the goal, the terminal transmits the information to the server.

[0557] 3. Data processing and calculation:

[0558] The server verifies the achievement information and checks it against a database.

[0559] After confirming that the objective has been achieved, reward information is generated.

[0560] 4. Output: Reward information

[0561] 5. Specific operation:

[0562] The server transmits the generated reward information to the terminal.

[0563] The terminal displays reward information to the user and notifies the user of the end of the game session.

[0564] Step 7: Incorporating the Emotion Engine

[0565] 1. Input: User's facial expression data, voice data

[0566] 2. Specific operation:

[0567] The emotion engine analyzes the user's facial expressions and voice in real time.

[0568] 3. Data processing and calculation:

[0569] The emotion engine generates the recognized emotion information and sends it to the server.

[0570] The server dynamically adjusts events and interactions based on the emotional information.

[0571] 4. Output: Coordinated event information, interaction information

[0572] 5. Specific operation:

[0573] The server transmits the adjusted event information to the terminal.

[0574] The device displays tailored event information to the user, personalizing the experience in the virtual space.

[0575] (Application example 2)

[0576] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0577] In current virtual stores, the experience is not fully personalized based on the user's emotions. This makes it difficult for users to smoothly browse the store and receive optimal product recommendations without feeling anxious or confused. Furthermore, there is a demand for a more immersive and interactive experience for users of the metaverse.

[0578] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0579] In this invention, the server includes means for transmitting authentication information for a user to log in to the metaverse, means for the server to receive the authentication information and authenticate the user, means for the server to generate a session ID for the authenticated user, means for the terminal to request scene data and character data from the server, means for the server to acquire the scene data and character data and send it to the terminal, means for the terminal to render the metaverse based on the scene data and character data received, means for the terminal to request a game objective from the server and display the received objective, means for the user to send a question to a character in the metaverse, and means for the server to generate a response to the question and send it to the terminal. The device includes a means for receiving a response from the user, a means for displaying the response received by the terminal to the user, a means for transmitting user behavior information to a server and checking an event occurrence condition, a means for generating event information and transmitting it to the terminal if the condition is met by the server, a means for displaying the event information received by the terminal to the user, a means for the server to verify that the user has achieved their goal and generate reward information, a means for transmitting the reward information to the terminal and displaying it, a means for capturing the user's facial expression with a camera mounted on the terminal, a means for providing an emotion engine that recognizes the user's emotion from the captured facial expression, a means for dynamically adjusting the experience in the metaverse based on the recognized emotion, and a means for the terminal to suggest and notify products based on the user's emotion. This dynamically optimizes the experience in the virtual store based on the user's emotion, enabling a more immersive shopping experience.

[0580] The "metaverse" refers to a virtual space built on the Internet where users can interact with other users and the environment in real time via avatars.

[0581] "Authentication information" refers to information such as a username and password used when a user accesses the metaverse, and is data used to verify the user's identity.

[0582] A "session ID" is a unique identifier that the server generates for an authenticated user and is used to manage the user's session.

[0583] "Scene data" is data that makes up the environment within the metaverse, and includes information such as three-dimensional landscapes, objects, and backgrounds.

[0584] "Character data" refers to information about the avatars used by users and NPCs (non-player characters) within the Metaverse, including their appearance, movements, attributes, etc.

[0585] "Rendering" refers to the process by which a computer visually depicts a three-dimensional environment within the Metaverse based on scene and character data.

[0586] "Game Objective" is information about a specific task or mission that a user must accomplish within the Metaverse.

[0587] A "question" is information that a user inquires about or verifies with a character in the metaverse.

[0588] A "response" is a response that the server generates in response to a user's question and that is provided to the user as part of the interaction.

[0589] "Behavioral information" is data on the actions a user takes within the metaverse, and is information that is used to check whether an event will occur.

[0590] "Event information" is data about specific occurrences that occur within the metaverse and are triggered by user actions or conditions.

[0591] "Reward information" is reward data generated by the server when a user achieves a game objective, and is provided to the user.

[0592] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[0593] "Dynamic adjustment" refers to the process of changing the experience within the metaverse in real time based on the user's current situation and emotions.

[0594] "Product suggestion" is the process of recommending appropriate products to users based on their emotions and behavior.

[0595] Basic system overview

[0596] The present invention is a system that utilizes an emotion engine to dynamically adjust a user's experience when they log in to a virtual store and engage in a purchasing experience. This system is composed of the following main components:

[0597] 1. User device: A device that accesses the metaverse and has a camera, such as smart glasses or a smartphone.

[0598] 2. Server system: Manages authentication information, provides scene data, generates questions and answers, and dynamically adjusts the experience.

[0599] 3. Metaverse Database: Stores scene data, character data, and product information within the Metaverse.

[0600] 4. Emotion Engine: Software that recognizes user emotions and dynamically adjusts the user experience based on that information.

[0601] Login process

[0602] The user accesses the virtual store using their own terminal and enters authentication information (user name and password) into the login form. The server receives this information and authenticates it by checking it against the metaverse database. If authentication is successful, the server generates a new session ID and returns it to the user's terminal. The terminal notifies the user that login was successful, and the entry point for the virtual store is displayed.

[0603] Loading a Virtual Store

[0604] After the user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data based on the session ID and sends it to the device. The device then renders a 3D virtual store environment based on the received data and displays it to the user.

[0605] Emotion-based product recommendations

[0606] A camera installed on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes them to recognize the user's emotions. The recognized emotion information is sent to the server, which dynamically adjusts product suggestions and notifications based on that information. For example, if the user is excited, the server will suggest limited-time offers. Conversely, if the user is confused, the server will display a support message.

[0607] Specific examples

[0608] As users browse products in the virtual store, a camera captures their facial expressions, and an emotion engine recognizes "excitement." The server receives this data and sends information about limited-time offers to the user's device, which then displays the information to the user.

[0609] Example prompts for generative AI models

[0610] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[0611] Hardware and software used

[0612] Hardware: Smart glasses, smartphone, on-board camera

[0613] Software: OpenCV (used for acquiring and preprocessing camera images), emotion_recognition (used for emotion recognition), virtual_shopping_platform (virtual store API)

[0614] In this way, a system is realized that dynamically optimizes the experience in the virtual store based on the user's emotions and personalizes the shopping experience.

[0615] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0616] Step 1:

[0617] A user accesses a virtual store using a terminal and enters authentication information (user name and password) into a login form. The terminal sends this authentication information to the server.

[0618] Input: Username and Password

[0619] Output: Sends authentication information to the server

[0620] What happens: The user enters their authentication information into a login form, and the device emails that information to the server.

[0621] Step 2:

[0622] The server verifies the user by checking the received authentication information against the metaverse database. If authentication is successful, the server generates a new session ID and sends it to the device.

[0623] Input: Credentials

[0624] Output: Session ID

[0625] Specific operation: The server performs a matching query against the database, and if authentication is successful, generates a session ID and sends it back to the device.

[0626] Step 3:

[0627] The terminal requests scene data and character data of the virtual store based on the session ID received from the server.

[0628] Input: Session ID

[0629] Output: Request for scene and character data

[0630] Specific operation: The terminal sends a request including the session ID to the server, requesting the necessary data.

[0631] Step 4:

[0632] The server retrieves scene data and character data from the metaverse database based on the session ID and sends them to the terminal.

[0633] Input: Session ID

[0634] Output: Scene data and character data

[0635] Specific operation: The server performs a database search, obtains scene data and character data, and sends them to the terminal.

[0636] Step 5:

[0637] The device renders a 3D environment of the virtual store based on the scene data and character data received and displays it to the user.

[0638] Input: Scene data and character data

[0639] Output: 3D environment of a virtual store

[0640] Specific operation: The device starts the rendering engine, draws the 3D environment based on the received data, and displays it on the screen.

[0641] Step 6:

[0642] The camera on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes the facial expressions to recognize the user's emotions. The recognized emotion information is then sent to the server.

[0643] Input: facial expression data

[0644] Output: Emotional information

[0645] Specific operation: The camera captures facial expressions, the emotion engine analyzes the facial expressions to recognize the emotions, and sends the information to the server.

[0646] Step 7:

[0647] The server dynamically suggests products suitable for the user based on the received emotion information, and sends the suggestions to the terminal.

[0648] Input: Emotion information

[0649] Output: Product proposal information

[0650] Specific operation: The server generates suggestions from a product database based on the emotion information and sends them to the device.

[0651] Step 8:

[0652] The terminal displays the product suggestion information received from the server to the user, and the user can view or purchase products according to the suggestion.

[0653] Input: Product proposal information

[0654] Output: Product display

[0655] Specific operation: The device displays the suggested information on the screen, and the user can browse or purchase.

[0656] The program and prompt to use

[0657] The programs used include "virtual_shopping_platform" and "emotion_recognition".

[0658] Example prompt sentence:

[0659] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[0660] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0661] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0662] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0663] [Second embodiment]

[0664] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0665] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0666] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0667] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0668] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0669] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0670] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0671] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0672] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0673] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0674] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0675] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0676] Basic system overview

[0677] The present invention is a system in which a user logs into the Metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. The operation of the system will be described in detail below for each processing step.

[0678] Overall system configuration

[0679] The system consists of the following main components:

[0680] 1. User Device

[0681] 2. Server System

[0682] 3. Metaverse Database

[0683] Login process

[0684] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[0685] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0686] Loading the Metaverse

[0687] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[0688] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[0689] Presentation of the game objective

[0690] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[0691] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0692] Cooperation and information gathering

[0693] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0694] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0695] Event Progression

[0696] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[0697] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[0698] Achieving goals and distributing rewards

[0699] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[0700] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[0701] The above is a specific description of the embodiment for carrying out the present invention.

[0702] The processing flow will be explained below.

[0703] Program processing steps

[0704] Step 1:

[0705] User Login

[0706] The user opens the login form using the terminal.

[0707] The terminal presents the user with a form for entering a username and password.

[0708] The user enters their authentication information and clicks the Login button.

[0709] The device sends the authentication information to the server.

[0710] The server checks the authentication information against a database and determines whether the authentication is successful.

[0711] If the server is successful in the authentication, it generates a session ID and sends it to the terminal.

[0712] The terminal displays a successful login message.

[0713] Step 2:

[0714] Loading the Metaverse

[0715] The terminal sends a request for scene data and character data to the server.

[0716] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[0717] The server acquires the character data and sends it to the device.

[0718] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[0719] Step 3:

[0720] Presentation of the game objective

[0721] The terminal requests game objective information from the server.

[0722] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[0723] The terminal displays the acquired game objective to the user.

[0724] Step 4:

[0725] Cooperation and information gathering

[0726] Users approach and interact with characters in the Metaverse.

[0727] The user inputs a question and sends it to the character.

[0728] The terminal transmits the query information to the server.

[0729] The server uses the character's AI engine to generate a response and send it to the device.

[0730] The terminal displays the response to the user.

[0731] Step 5:

[0732] Event Progression

[0733] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[0734] The server receives the user's behavior information and checks the event occurrence conditions.

[0735] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[0736] The device displays event information to the user and prompts them to take the next action.

[0737] Step 6:

[0738] Achieving goals and distributing rewards

[0739] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[0740] The server verifies the achievement of the objectives and generates reward information.

[0741] The server transmits the reward information to the terminal.

[0742] The terminal displays reward information to the user and notifies them that the game session is ending.

[0743] Specific example explanation

[0744] As a concrete example, we will explain the specific actions that "User A" takes from logging in to achieving his or her goal.

[0745] Step 1:

[0746] "User A" uses a terminal to open the login form page, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which checks the authentication information in the database. If the authentication is successful, the server generates a session ID and sends it back to the terminal. "User A"'s terminal displays a login success message.

[0747] Step 2:

[0748] After "User A" successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database and sends it to "User A's" device. The device renders the metaverse based on the received data and displays it to "User A."

[0749] Step 3:

[0750] "User A" operates the device to confirm the objective of the game, and the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays the information to "User A" and presents the objective of "find the lost treasure."

[0751] Step 4:

[0752] "User A" approaches a villager character (CP) and asks "Where is the treasure?" The terminal of "User A" sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the terminal. The terminal displays the response to "User A."

[0753] Step 5:

[0754] When "User A" heads towards the castle, the device sends this action information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates event information that "the castle gates have been magically closed" and sends it to the device. The device then displays this event information to "User A" and prompts him to take the next action.

[0755] Step 6:

[0756] "User A" collects the necessary items and finds the treasure. The device sends the information to the server, which verifies that the objective was achieved. The server generates reward information (e.g., in-game currency) and sends it to the device. The device displays the reward information to "User A" and notifies him or her that the game session is ending.

[0757] Example 1

[0758] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0759] Current virtual reality systems require complex system processing after a user logs in, which can degrade the quality of the user experience. Furthermore, when users interact with characters in the virtual space, conventional systems only provide limited responses, resulting in a lack of completeness as a comprehensive gaming environment. Furthermore, there are many instances where manual processing is required for reward distribution when a user achieves a goal, and automation is needed.

[0760] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0761] In this invention, the server includes means for receiving and verifying authentication information for a user to log in to a virtual space, means for generating a session ID for the authenticated user, and means for generating question and answer prompts for a character in the virtual space using a generative AI model. This simplifies system processing after a user logs in, improving the quality of the user experience. It also enables more diverse and realistic responses in character interactions in the virtual space, improving the overall game environment. Furthermore, it automates reward allocation upon goal achievement, increasing overall efficiency.

[0762] "Authentication information" refers to a combination of a username and password that a user uses to log in to a virtual space.

[0763] A "server" is a central computer system for the entire virtual space system, providing data storage, processing, user authentication, and other functions.

[0764] A "session ID" is a unique identifier generated by the server to identify a user after the user has been authenticated.

[0765] A "terminal" is a device used by a user to access a virtual space, and includes a PC, smartphone, tablet, etc.

[0766] "Scene data" is data that constitutes the environment, background, objects, etc. within a virtual space.

[0767] "Character data" refers to data that includes information, attributes, actions, etc. of characters in a virtual space.

[0768] "Rendering" is the process of visually displaying a virtual 3D environment based on scene data and character data.

[0769] The "objective of the game" is the goal or task that the user must achieve within the virtual space.

[0770] A "generative AI model" is an algorithm or model that uses artificial intelligence technology to generate appropriate responses or content based on user input.

[0771] "Behavioral information" is data on the actions and choices made by the user within the virtual space.

[0772] An "event occurrence condition" is a condition or trigger element necessary for a specific event to occur in virtual space.

[0773] "Event information" is information that is generated when an event occurrence condition is met, and includes details of the event and the next action to be taken.

[0774] "Reward information" is details of the rewards a user will receive when they achieve a goal in the game. Examples include virtual currency, items, etc.

[0775] The present invention is a system in which a user logs into a virtual space and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Each component and its operation will be described in detail below.

[0776] User terminal

[0777] User devices include PCs, smartphones, tablets, and other devices. Users use these devices to access the virtual space and perform everything from login to gameplay. The devices use software such as HTML, JavaScript, React, and Three.js to execute the user interface and 3D rendering.

[0778] Server System

[0779] The server system is a central computer system that handles user authentication, database management, response generation using generative AI models, event management, etc. The server runs on software such as Node.js and MySQL.

[0780] Metaverse Database

[0781] The metaverse database stores user information, scene data, character data, game objectives, event information, etc. MySQL is used as the database management system.

[0782] Login process

[0783] The user displays a login form on their device and enters their username and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a new session ID and returns it to the device. The device notifies the user of successful authentication and displays an entry point to the virtual space.

[0784] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0785] Loading the Metaverse

[0786] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders a 3D virtual environment based on the received data and displays it to the user.

[0787] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the virtual space based on that data.

[0788] Presentation of the game objective

[0789] The device requests game objective information from the server. The server retrieves the objective information from a database based on the user's session ID. The server sends the objective information to the device, which displays it to the user.

[0790] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0791] Cooperation and information gathering

[0792] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine (generative AI model) to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0793] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[0794] Prompt Sentence Examples

[0795] User prompt: "Where is the treasure?"

[0796] Event Progression

[0797] When a user takes a specific action (e.g., heads to the castle), the device sends the action information to the server. The server checks the event occurrence conditions and generates event information if the conditions are met. The server then sends the event information to the device, which then displays it to the user.

[0798] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[0799] Achieving goals and distributing rewards

[0800] When the user achieves the game objective, the device sends that information to the server. The server confirms the achievement and generates reward information (e.g., in-game currency). The server then sends the reward information to the device, which displays it to the user. This ends the game session.

[0801] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information, and sends it to the device. The device displays it to "User A" and notifies him that the game is over.

[0802] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0803] Step 1:

[0804] The user enters their username and password into the login form on their device.

[0805] Input: Username, Password

[0806] Specific behavior: The device displays the login screen using HTML and JavaScript. The user fills out the form and clicks the login button.

[0807] Step 2:

[0808] The terminal transmits the entered authentication information to the server.

[0809] Input: Username, Password

[0810] Specific operation: Sends authentication information to the server via an HTTP POST request using the JavaScript Fetch API.

[0811] Step 3:

[0812] The server checks the credentials against a database.

[0813] Input: Username, Password

[0814] Output: Authentication success or failure, session ID

[0815] What happens: The server uses Node.js to query the MySQL database for authentication information. If the credentials match, it generates a new session ID with a UUID.

[0816] Step 4:

[0817] The server returns the authentication result to the terminal and includes the new session ID.

[0818] Input: Authentication success or failure, session ID

[0819] Output: Session ID

[0820] Specific operation: If authentication is successful, the session ID is returned to the terminal in the HTTP response.

[0821] Step 5:

[0822] The terminal notifies the user of successful authentication and displays an entry point to the virtual space.

[0823] Input: Session ID

[0824] Specific behavior: The device displays a successful authentication message and an entry point link. The screen is updated using React.

[0825] Step 6:

[0826] The user clicks on an entry point into the virtual space.

[0827] Input: Entry point URL, Session ID

[0828] Specific operation: The terminal sends a request to the entry point to the server.

[0829] Step 7:

[0830] The terminal sends a request for scene data and character data to the server.

[0831] Input: Session ID

[0832] Specific operation: The device requests scene data and character data from the server using an HTTP GET request.

[0833] Step 8:

[0834] The server retrieves scene data and character data from the database and transmits them to the terminal.

[0835] Input: Session ID

[0836] Output: Scene data, character data

[0837] Specific operation: The server uses Node.js to retrieve the necessary data from the MySQL database and send it to the terminal as an HTTP response.

[0838] Step 9:

[0839] The device renders a 3D virtual environment based on the scene data and character data it receives and displays it to the user.

[0840] Input: Scene data, character data

[0841] What it does: The device uses Three.js to render the scene and characters in 3D and display them in the user interface.

[0842] Step 10:

[0843] The terminal requests information for game purposes from the server.

[0844] Input: Session ID

[0845] Specific operation: The device requests the game objective from the server via an HTTP GET request.

[0846] Step 11:

[0847] The server retrieves game objective information from the database based on the user's session ID and sends it to the terminal.

[0848] Input: Session ID

[0849] Output: Game objective information

[0850] Specific operation: The server uses Node.js to retrieve game objective information from the MySQL database and sends it to the terminal as an HTTP response.

[0851] Step 12:

[0852] The terminal displays the received game objective information to the user.

[0853] Input: Game objective information

[0854] Specific operation: The device uses React to display game objective information on the user interface.

[0855] Step 13:

[0856] The user sends a question to a character in the virtual space.

[0857] Input: User's question

[0858] Specific operation: The device sends the question to the server.

[0859] Step 14:

[0860] The terminal sends the user's question to the server.

[0861] Input: User's question

[0862] Specific operation: The question is sent to the server via an HTTP POST request, possibly using WebSocket.

[0863] Step 15:

[0864] The server uses an AI engine (generative AI model) to generate a response to the question and send it to the terminal.

[0865] Input: User's question

[0866] Output: Response by AI engine

[0867] Specific operation: The server uses a generative AI model (e.g., GPT-3) to generate an answer to the question and sends it to the device in an HTTP response.

[0868] Step 16:

[0869] The terminal displays the received response to the user.

[0870] Input: Response by AI engine

[0871] Specific behavior: The device displays the response in the user interface using React.

[0872] Step 17:

[0873] The user takes a specific action (e.g., heads to the castle).

[0874] Input: User behavior information

[0875] Specific operation: The device sends its behavior information to the server.

[0876] Step 18:

[0877] The terminal transmits the user's behavior information to the server.

[0878] Input: User behavior information

[0879] Specific operation: Sends behavioral information to the server via an HTTP POST request.

[0880] Step 19:

[0881] The server checks the event occurrence conditions, and if the conditions are met, generates event information and sends it to the terminal.

[0882] Input: User behavior information

[0883] Output: Event information

[0884] Specific operation: The server checks the event occurrence conditions based on the behavioral information, and if the conditions are met, it generates event information using Node.js and sends it to the terminal as an HTTP response.

[0885] Step 20:

[0886] The terminal displays the received event information to the user.

[0887] Input: Event information

[0888] Specific behavior: The device displays the event information on the user interface using React.

[0889] Step 21:

[0890] The user completes the game objective (e.g., finds treasure).

[0891] Input: Objective achievement information

[0892] Specific operation: The device sends the information to the server.

[0893] Step 22:

[0894] The terminal transmits the goal achievement information to the server.

[0895] Input: Objective achievement information

[0896] Specific behavior: Sends achievement information to the server using an HTTP POST request.

[0897] Step 23:

[0898] The server verifies the achievement of the objectives and generates reward information.

[0899] Input: Objective achievement information

[0900] Output: Reward information

[0901] Specific operation: The server confirms that the goal has been achieved and generates reward information using Node.js.

[0902] Step 24:

[0903] The server transmits the reward information to the terminal.

[0904] Input: Reward information

[0905] Specific operation: Reward information is sent to the terminal in an HTTP response.

[0906] Step 25:

[0907] The terminal displays the received reward information to the user.

[0908] Input: Reward information

[0909] Specific behavior: The device displays the reward information on the user interface using React and notifies the end of the game session.

[0910] (Application example 1)

[0911] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] There is a need for an effective method to facilitate interactions with characters and event progression in order for users to achieve specific goals in the metaverse. There is also a need to make the shopping experience more attractive by allowing users to receive special benefits such as coupons when purchasing specific products in virtual stores.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0914] In this invention, the server includes: means for sending authentication information for a user to log in to the metaverse; means for the server to receive the authentication information and authenticate the user; means for the server to generate a session ID for the authenticated user; means for the terminal to request scene data and character data from the server; means for the server to acquire the scene data and character data and send it to the terminal; means for the terminal to render the metaverse based on the scene data and character data received; means for the terminal to request a game objective from the server and display the received objective; means for the user to send a question to a character in the metaverse; means for the server to generate a response to the question and send it to the terminal; means for the terminal to display the response received by the user; means for sending user behavior information to the server and confirming the event occurrence conditions; means for the server to generate event information and send it to the terminal if the conditions are met; means for the server to display the event information received by the terminal to the user; means for the server to verify the user's achievement of the objective and generate reward information; means for sending the reward information to the terminal and displaying it; means for generating a coupon when the user purchases a specific product and sending it to the terminal; and means for receiving a prompt message regarding the purchase of a specific product using a generative AI model. This allows users to smoothly interact with and progress through events within the metaverse, and also makes it possible to offer coupons when purchasing certain products.

[0915] The "metaverse" is a three-dimensional virtual environment in which users can interact with other users and virtual characters within a virtual space.

[0916] "Authentication information" refers to information such as a username and password that a user uses when accessing the metaverse.

[0917] A "session ID" is a unique identification number generated by the server for an authenticated user.

[0918] "Scene data" is data about the environment and objects within the metaverse.

[0919] "Character data" refers to data about characters and NPCs (Non-Player Characters) controlled by users within the Metaverse.

[0920] "Rendering" is the process by which a computer generates a visually displayable image from three-dimensional scene data.

[0921] A "coupon" is electronic information indicating a discount or benefit that is offered when a user meets certain conditions.

[0922] A "prompt sentence" is textual input information that a generative AI model uses to generate a response.

[0923] A "generative AI model" is an artificial intelligence model that generates an appropriate response based on a given prompt sentence.

[0924] "Event information" is information about specific actions or events that occur within the metaverse in response to user behavior conditions.

[0925] "Reward information" is information regarding rewards that will be provided to users when they achieve specific goals within the metaverse.

[0926] A "virtual store" is a store that exists virtually on the Internet, and is a virtual space where users can gather product information and make purchases.

[0927] Basic system overview

[0928] The present invention is a system that allows users to log in to a virtual store and collect product information to achieve specific goals, or to receive rewards such as coupons as they progress through events. The operation of the system is described in detail below.

[0929] Overall system configuration

[0930] The system mainly consists of the following major components:

[0931] 1. User device: Smartphone application, etc.

[0932] 2. Server system: A server that processes and manages data.

[0933] 3. Database: A database that manages metaverse scene data, character data, product data, etc.

[0934] 4. Generative AI model: An artificial intelligence model that generates appropriate responses or coupons based on user prompts.

[0935] Login process

[0936] The user accesses the virtual store using their terminal. A login form is displayed, the user enters their authentication information (username and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database. If authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authorized and the entry point to the virtual store is displayed.

[0937] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends "User A's" authentication information to the server, which then verifies it in the database, generates a session ID, and sends it to "User A's" terminal.

[0938] Loading a Virtual Store

[0939] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the terminal. The terminal then renders a 3D environment of the virtual store based on this data and displays it to the user.

[0940] As a concrete example, the terminal of "User A" requests scene data and product data from the server, and the server retrieves this data from the database and sends it to the terminal of "User A." The terminal then displays a virtual store based on that data.

[0941] State your shopping purpose

[0942] The terminal requests shopping purpose information from the server, and the server obtains the purpose information based on the user's session ID and sends it to the terminal, which then displays it to the user, allowing them to take the first step toward achieving their purpose.

[0943] For example, "User A" sends a request to the server to confirm his / her shopping purpose on his / her device, and the server acquires the purpose information "to find discount sales" and sends it to "User A's" device. The device displays it, allowing "User A" to recognize his / her purpose.

[0944] Cooperation and information gathering

[0945] Users can interact with store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question, the device sends the question to the server. The server uses a generative AI model to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[0946] For example, "User A" asks a store clerk character for "details of the most popular item." The device sends the question to the server, and the server generates a response saying, "This item is currently on sale, 20% off," and sends it to the device. The device displays it to "User A."

[0947] Event Progression

[0948] When a user purchases a specific product, the behavioral information is sent from the device to the server. The server checks the purchase conditions, and if the conditions are met, the server generates event information and sends it to the device. The device displays the event information to the user, prompting them to take the next action.

[0949] For example, when "User A" places a specific product in their cart and purchases it, their device sends that information to the server. The server confirms that the conditions are met, generates event information for a "10% off coupon for your next purchase," and sends it to their device. The device then displays this to "User A."

[0950] Achieving goals and distributing rewards

[0951] When the user achieves their shopping goal, the terminal sends the information to the server. The server verifies the goal achievement and generates reward information and sends it to the terminal. The terminal displays the reward information to the user, and the shopping session ends.

[0952] As a concrete example, when "User A" achieves his / her goal in a virtual store, the device sends this information to the server. The server confirms that the goal has been achieved, generates a "coupon for next use" and sends it to the device. The device displays this to "User A" and notifies him / her that his / her shopping has finished.

[0953] An example prompt is:

[0954] "Generate a 10% off coupon for the next purchase by the user when they purchase a specific product."

[0955] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0956] Step 1:

[0957] A user accesses a virtual store from a terminal, enters authentication information (user name and password) in the login form, and presses the login button. The entered authentication information is sent from the terminal to the server. The server compares the received authentication information against a database and performs authentication. If authentication is successful, the server generates a new session ID and sends it back to the terminal. The terminal then notifies the user that access to the system has been permitted, and displays the entry point to the virtual store.

[0958] Input: Authentication information (username, password)

[0959] Output: Session ID

[0960] Step 2:

[0961] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves the metaverse scene data and product data from the database based on the session ID and sends them to the terminal. The terminal then renders the 3D environment of the virtual store based on this data and displays it to the user.

[0962] Input: Session ID

[0963] Output: Scene data and product data

[0964] Step 3:

[0965] The terminal sends a request for shopping intent information to the server. The server retrieves the intent information from the database based on the session ID and sends it to the terminal. The terminal displays the intent information to the user and prompts them to take their first action.

[0966] Input: Session ID

[0967] Output: Shopping intent information

[0968] Step 4:

[0969] Users can ask questions to store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question such as "Details of the most popular item," the device sends the question to the server. The server uses a generative AI model to generate an appropriate response to the question and sends it to the device. The device then displays the received response to the user.

[0970] Input: User prompt

[0971] Output: Response

[0972] Step 5:

[0973] When a user places a specific product in their cart and purchases it, that behavioral information is sent from the device to the server. The server checks the purchase conditions based on this information, and if the conditions are met, generates event information and sends it to the device. For example, if the conditions are met, the server generates a "10% off coupon for your next purchase" and sends it to the device. The device then displays this coupon information to the user.

[0974] Input: Purchase Information

[0975] Output: Coupon information

[0976] Step 6:

[0977] When the user achieves their shopping goal, the terminal sends the information to the server, which verifies the goal achievement, generates reward information, and sends it to the terminal, which displays the reward information to the user and notifies them that the shopping session has ended.

[0978] Input: Objective achievement information

[0979] Output: Reward information

[0980] The above are the specific processing steps of the system of the present invention. This allows users to smoothly interact with the virtual store and progress through events. Furthermore, coupons can be obtained when purchasing specific products, which is expected to improve the shopping experience.

[0981] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0982] Basic system overview

[0983] The present invention is a system in which a user logs into the metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience within the metaverse based on the user's emotions.

[0984] Overall system configuration

[0985] The system consists of the following main components:

[0986] 1. User Device

[0987] 2. Server System

[0988] 3. Metaverse Database

[0989] 4. Emotion Engine

[0990] Login process

[0991] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[0992] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[0993] Loading the Metaverse

[0994] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[0995] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[0996] Presentation of the game objective

[0997] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[0998] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[0999] Cooperation and information gathering

[1000] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1001] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1002] Event Progression

[1003] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[1004] As a concrete example, when "User A" heads to the castle, the device sends this behavioral information to the server. The server confirms that the conditions are met, generates event information indicating that the castle gate has been magically closed, and sends it to the device. The device then displays this event information to "User A."

[1005] Achieving goals and distributing rewards

[1006] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[1007] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[1008] Incorporating an emotion engine

[1009] The emotion engine analyzes the user's facial expressions and voice to recognize their emotions, which are then sent to the server, which then dynamically adjusts events and interactions within the metaverse.

[1010] As a concrete example, when "User A" talks to a villager character in the metaverse, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays it to "User A."

[1011] The introduction of the emotion engine personalizes the gaming experience based on the real-time emotions felt by the user, providing an even more immersive experience.

[1012] The processing flow will be explained below.

[1013] Program processing steps

[1014] Step 1:

[1015] User Login

[1016] The user opens the login form using the terminal.

[1017] The terminal presents the user with a form for entering a username and password.

[1018] The user enters their authentication information and clicks the Login button.

[1019] The device sends the authentication information to the server.

[1020] The server checks the authentication information against a database and determines whether the authentication is successful.

[1021] If the authentication is successful, the server generates a session ID and sends it to the terminal.

[1022] The terminal displays a successful login message.

[1023] Step 2:

[1024] Loading the Metaverse

[1025] The terminal sends a request for scene data and character data to the server.

[1026] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[1027] The server also obtains character data and sends it to the device.

[1028] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[1029] Step 3:

[1030] Early recognition of emotions

[1031] When a user logs into the metaverse on their device, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotions.

[1032] The emotion engine transmits the recognized emotion information to the server.

[1033] The server stores the emotional information in a database and uses it in subsequent gameplay.

[1034] Step 4:

[1035] Presentation of the game objective

[1036] The terminal requests game objective information from the server.

[1037] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[1038] The terminal displays the acquired game objective to the user.

[1039] Step 5:

[1040] Cooperation and information gathering

[1041] Users approach and interact with characters in the Metaverse.

[1042] The user inputs a question and sends it to the character.

[1043] The terminal transmits the query information to the server.

[1044] The server uses the character's AI engine to generate a response and send it to the device.

[1045] The terminal displays the response to the user.

[1046] Step 6:

[1047] Emotional remonitoring

[1048] The emotion engine periodically analyzes the user's facial expressions and voice, recognizing emotions in real time.

[1049] The recognized emotion information is sent back to the server.

[1050] The server dynamically adjusts events and character responses based on the emotional information.

[1051] Step 7:

[1052] Event Progression

[1053] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[1054] The server receives the user's behavior information and checks the event occurrence conditions.

[1055] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[1056] The device displays event information to the user and prompts them to take the next action.

[1057] Step 8:

[1058] Achieving goals and distributing rewards

[1059] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[1060] The server verifies the achievement of the objectives and generates reward information.

[1061] The server transmits the reward information to the terminal.

[1062] The terminal displays reward information to the user and notifies them that the game session is ending.

[1063] Examples of emotion engines

[1064] As a concrete example, we will explain the process by which "User A" logs in to the Metaverse and plays.

[1065] Step 1:

[1066] "User A" opens the login form on the terminal, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which verifies it against the database. If authentication is successful, the server sends a session ID, and the terminal displays a login success message.

[1067] Step 2:

[1068] After "User A" logs in, the device sends a request for scene and character data to the server. The server retrieves the data and sends it to the device. The device renders the metaverse and displays it to the user.

[1069] Step 3:

[1070] When "User A" enters the metaverse, the device's built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotions and transmits that information to the server, which stores the emotion information and uses it for subsequent processing.

[1071] Step 4:

[1072] In order for "User A" to confirm the objective of the game, the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays it to "User A" and presents the objective of "find the treasure."

[1073] Step 5:

[1074] "User A" asks a villager character "Where is the treasure?" The device sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the device. The device displays it to "User A."

[1075] Step 6:

[1076] While "User A" is interacting with a villager, the emotion engine recognizes tension from "User A's" facial expression. The server receives this information and adjusts the villager character's response to encourage relaxation. The device then converts the villager character's response into a suggestion for "what to do to relax" and displays it to the user.

[1077] Step 7:

[1078] When "User A" heads to the castle, the device sends that behavioral information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates new event information stating that "the castle gates have been magically closed" and sends it to the device. The device displays this to "User A" and prompts them to take the next action.

[1079] Step 8:

[1080] When "User A" finds the treasure, the device sends the achievement information to the server. The server confirms the achievement, generates reward information (e.g., in-game currency), and sends it to the device. The device displays the information to "User A" and notifies him or her that the game session is ending.

[1081] Example 2

[1082] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1083] Conventional virtual reality systems lack the ability to recognize users' emotions in real time and dynamically adjust the experience based on those emotions, which results in a lack of immersion and satisfaction for users.

[1084] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a session ID for an authenticated user, a means for acquiring scene data and character data and transmitting them to the terminal, and a means for transmitting user behavior information to the server and checking the event occurrence conditions. This allows the user's emotions to be recognized in real time after logging in, and the experience in the virtual space to be dynamically adjusted.

[1085] "Authentication information" is data such as a username and password that a user uses to access a system.

[1086] A "session ID" is a unique identifier generated by the server to identify operations by an authenticated user.

[1087] "Scene data" is data that includes information about the environment and objects in a virtual space.

[1088] "Character data" is data that includes information about characters in a virtual space.

[1089] A "virtual space" is a computer-generated 3D environment that can be accessed and experienced by a user.

[1090] "Rendering" is the process by which a computer generates a graphical display based on scene data and character data.

[1091] "Game objective" refers to the tasks or goals that a user must accomplish within the virtual space.

[1092] "Event information" is information about an event that occurs when certain conditions are met in the virtual space.

[1093] "Reward information" is information about the reward that a user can receive when they achieve the objective of the game.

[1094] An "emotion engine" is software that recognizes emotions in real time from a user's facial expressions, voice, etc.

[1095] A "virtual character" is a computer-generated character that interacts with a user in a virtual space.

[1096] "Interaction" refers to the actions or operations that a user performs to interact with elements in a virtual space.

[1097] Basic system overview

[1098] The present invention is a system in which a user logs into a virtual space and cooperates with other characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience in the virtual space based on the user's emotions.

[1099] Overall system configuration

[1100] The system consists of the following main components:

[1101] 1. User Device

[1102] 2. Server System

[1103] 3. Virtual Space Database

[1104] 4. Emotion Engine

[1105] Hardware and software used

[1106] User device: Computer or smartphone (e.g., Windows PC, smartphone)

[1107] Server systems: high-performance servers (e.g. cloud infrastructure)

[1108] Virtual spatial database: Data management system (e.g., MySQL, NoSQL database)

[1109] Emotion engine: Emotion recognition software (e.g., Emotion SDK)

[1110] Login process

[1111] A user accesses the virtual space system using their own terminal. A login form is displayed, and the user enters authentication information (user name and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authenticated, and the entry point to the virtual space is displayed. As a concrete example, "User A" attempts to log in on the terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1112] Loading the virtual world

[1113] When a user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders and displays a 3D environment in the virtual space based on this data. As a concrete example, after "User A" has successfully logged in, the device requests scene data and character information from the server. The server retrieves this data and sends it to "User A's" device, and the device then displays the virtual space based on that data.

[1114] Presentation of the game objective

[1115] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays this to the user, allowing them to take the first step towards achieving their objective. As a concrete example, when "User A" wants to confirm the game objective, the device sends a request to the server. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1116] Cooperation and information gathering

[1117] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device displays the response to the user. As a concrete example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1118] Event Progression

[1119] When a user takes a specific action (for example, heading towards the castle), the device sends that action information to the server, and the server checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device displays that event information to the user and prompts them to take the next action. As a specific example, when "User A" heads towards the castle, the device sends that action information to the server. The server checks that the conditions are met, generates event information that the castle gates have been magically closed, and sends it to the device. The device displays that event information to "User A."

[1120] Achieving goals and distributing rewards

[1121] When a user achieves the game objective, the device sends that information to the server. The server verifies that the objective has been achieved, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends. As a specific example, when "User A" finds treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays it to "User A," notifying him that the game is over.

[1122] Incorporating an emotion engine

[1123] The emotion engine recognizes emotions by analyzing the user's facial expressions and voice. The recognized emotional information is sent to the server, which then dynamically adjusts events and interactions in the virtual space based on that information. As a specific example, when "User A" talks to a villager character in the virtual space, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays this to "User A." The introduction of the emotion engine personalizes the game experience based on the user's real-time emotions, providing an even more immersive experience.

[1124] Prompt Sentence Examples

[1125] "User A logs into a virtual world and works with his friends to find treasure. On the way to the castle, the emotion engine recognizes the user's stress, and a villager character suggests ways for the user to relax. Please explain this scenario in natural language."

[1126] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1127] Step 1: Login process

[1128] 1. Input: Authentication information (user name, password)

[1129] 2. Specific operation:

[1130] The user enters their username and password into the login form.

[1131] The terminal transmits the entered authentication information to the server.

[1132] 3. Data processing and calculation:

[1133] The server checks the received authentication information against its database.

[1134] A database query is performed to verify that the username and password match.

[1135] 4. Output: Authentication result, session ID (if successful)

[1136] 5. Specific operation:

[1137] If the authentication is successful, the server generates a new session ID and sends it to the terminal.

[1138] The terminal receives the session ID and notifies the user that the login was successful.

[1139] Step 2: Loading the Virtual Space

[1140] 1. Input: Session ID

[1141] 2. Specific operation:

[1142] After the user has successfully logged in, the terminal sends a request for scene data and character data to the server.

[1143] 3. Data processing and calculation:

[1144] The server retrieves scene data and character data from the database based on the session ID.

[1145] Execute database queries and extract the required data.

[1146] 4. Output: Scene data, character data

[1147] 5. Specific operation:

[1148] The server transmits the acquired data to the terminal.

[1149] The device renders a 3D environment based on the data it receives and displays the virtual space.

[1150] Step 3: Present the objective of the game

[1151] 1. Input: Session ID

[1152] 2. Specific operation:

[1153] The terminal requests game objective information from the server.

[1154] 3. Data processing and calculation:

[1155] The server retrieves the desired information from the database based on the session ID.

[1156] Execute a database query to extract the desired information.

[1157] 4. Output: Game objective information

[1158] 5. Specific operation:

[1159] The server transmits the target information to the terminal.

[1160] The terminal displays the received objective information to the user and guides the user to the next step.

[1161] Step 4: Collaborate and gather information

[1162] 1. Input: User's question

[1163] 2. Specific operation:

[1164] The user inputs a question to the character and sends it.

[1165] The device sends the question to the server.

[1166] 3. Data processing and calculation:

[1167] The server uses a generative AI model to generate answers to questions.

[1168] Feed the AI ​​model with questions and generate appropriate responses.

[1169] 4. Output: Response from the character

[1170] 5. Specific operation:

[1171] The server generates a response and sends it to the terminal.

[1172] The terminal displays the response to the user.

[1173] Step 5: Event Progression

[1174] 1. Input: User behavior information

[1175] 2. Specific operation:

[1176] The user takes a specific action (e.g., heads to the castle).

[1177] The terminal transmits the behavior information to the server.

[1178] 3. Data processing and calculation:

[1179] The server executes a database query to check the conditions under which the event occurs.

[1180] If the condition is met, new event information is generated.

[1181] 4. Output: Event information

[1182] 5. Specific operation:

[1183] The server transmits the event information to the terminal.

[1184] The device displays the event information to the user and guides them on the next action.

[1185] Step 6: Achieving Objectives and Distributing Rewards

[1186] 1. Input: Objective achievement information

[1187] 2. Specific operation:

[1188] When the user achieves the goal, the terminal transmits the information to the server.

[1189] 3. Data processing and calculation:

[1190] The server verifies the achievement information and checks it against a database.

[1191] After confirming that the objective has been achieved, reward information is generated.

[1192] 4. Output: Reward information

[1193] 5. Specific operation:

[1194] The server transmits the generated reward information to the terminal.

[1195] The terminal displays reward information to the user and notifies the user of the end of the game session.

[1196] Step 7: Incorporating the Emotion Engine

[1197] 1. Input: User's facial expression data, voice data

[1198] 2. Specific operation:

[1199] The emotion engine analyzes the user's facial expressions and voice in real time.

[1200] 3. Data processing and calculation:

[1201] The emotion engine generates the recognized emotion information and sends it to the server.

[1202] The server dynamically adjusts events and interactions based on the emotional information.

[1203] 4. Output: Coordinated event information, interaction information

[1204] 5. Specific operation:

[1205] The server transmits the adjusted event information to the terminal.

[1206] The device displays tailored event information to the user, personalizing the experience in the virtual space.

[1207] (Application example 2)

[1208] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1209] In current virtual stores, the experience is not fully personalized based on the user's emotions. This makes it difficult for users to smoothly browse the store and receive optimal product recommendations without feeling anxious or confused. Furthermore, there is a demand for a more immersive and interactive experience for users of the metaverse.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1211] In this invention, the server includes means for transmitting authentication information for a user to log in to the metaverse, means for the server to receive the authentication information and authenticate the user, means for the server to generate a session ID for the authenticated user, means for the terminal to request scene data and character data from the server, means for the server to acquire the scene data and character data and send it to the terminal, means for the terminal to render the metaverse based on the scene data and character data received, means for the terminal to request a game objective from the server and display the received objective, means for the user to send a question to a character in the metaverse, and means for the server to generate a response to the question and send it to the terminal. The device includes a means for receiving a response from the user, a means for displaying the response received by the terminal to the user, a means for transmitting user behavior information to a server and checking an event occurrence condition, a means for generating event information and transmitting it to the terminal if the condition is met by the server, a means for displaying the event information received by the terminal to the user, a means for the server to verify that the user has achieved their goal and generate reward information, a means for transmitting the reward information to the terminal and displaying it, a means for capturing the user's facial expression with a camera mounted on the terminal, a means for providing an emotion engine that recognizes the user's emotion from the captured facial expression, a means for dynamically adjusting the experience in the metaverse based on the recognized emotion, and a means for the terminal to suggest and notify products based on the user's emotion. This dynamically optimizes the experience in the virtual store based on the user's emotion, enabling a more immersive shopping experience.

[1212] The "metaverse" refers to a virtual space built on the Internet where users can interact with other users and the environment in real time via avatars.

[1213] "Authentication information" refers to information such as a username and password used when a user accesses the metaverse, and is data used to verify the user's identity.

[1214] A "session ID" is a unique identifier that the server generates for an authenticated user and is used to manage the user's session.

[1215] "Scene data" is data that makes up the environment within the metaverse, and includes information such as three-dimensional landscapes, objects, and backgrounds.

[1216] "Character data" refers to information about the avatars used by users and NPCs (non-player characters) within the Metaverse, including their appearance, movements, attributes, etc.

[1217] "Rendering" refers to the process by which a computer visually depicts a three-dimensional environment within the Metaverse based on scene and character data.

[1218] "Game Objective" is information about a specific task or mission that a user must accomplish within the Metaverse.

[1219] A "question" is information that a user inquires about or verifies with a character in the metaverse.

[1220] A "response" is a response that the server generates in response to a user's question and that is provided to the user as part of the interaction.

[1221] "Behavioral information" is data on the actions a user takes within the metaverse, and is information that is used to check whether an event will occur.

[1222] "Event information" is data about specific occurrences that occur within the metaverse and are triggered by user actions or conditions.

[1223] "Reward information" is reward data generated by the server when a user achieves a game objective, and is provided to the user.

[1224] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[1225] "Dynamic adjustment" refers to the process of changing the experience within the metaverse in real time based on the user's current situation and emotions.

[1226] "Product suggestion" is the process of recommending appropriate products to users based on their emotions and behavior.

[1227] Basic system overview

[1228] The present invention is a system that utilizes an emotion engine to dynamically adjust a user's experience when they log in to a virtual store and engage in a purchasing experience. This system is composed of the following main components:

[1229] 1. User device: A device that accesses the metaverse and has a camera, such as smart glasses or a smartphone.

[1230] 2. Server system: Manages authentication information, provides scene data, generates questions and answers, and dynamically adjusts the experience.

[1231] 3. Metaverse Database: Stores scene data, character data, and product information within the Metaverse.

[1232] 4. Emotion Engine: Software that recognizes user emotions and dynamically adjusts the user experience based on that information.

[1233] Login process

[1234] The user accesses the virtual store using their own terminal and enters authentication information (user name and password) into the login form. The server receives this information and authenticates it by checking it against the metaverse database. If authentication is successful, the server generates a new session ID and returns it to the user's terminal. The terminal notifies the user that login was successful, and the entry point for the virtual store is displayed.

[1235] Loading a Virtual Store

[1236] After the user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data based on the session ID and sends it to the device. The device then renders a 3D virtual store environment based on the received data and displays it to the user.

[1237] Emotion-based product recommendations

[1238] A camera installed on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes them to recognize the user's emotions. The recognized emotion information is sent to the server, which dynamically adjusts product suggestions and notifications based on that information. For example, if the user is excited, the server will suggest limited-time offers. Conversely, if the user is confused, the server will display a support message.

[1239] Specific examples

[1240] As users browse products in the virtual store, a camera captures their facial expressions, and an emotion engine recognizes "excitement." The server receives this data and sends information about limited-time offers to the user's device, which then displays the information to the user.

[1241] Example prompts for generative AI models

[1242] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[1243] Hardware and software used

[1244] Hardware: Smart glasses, smartphone, on-board camera

[1245] Software: OpenCV (used for acquiring and preprocessing camera images), emotion_recognition (used for emotion recognition), virtual_shopping_platform (virtual store API)

[1246] In this way, a system is realized that dynamically optimizes the experience in the virtual store based on the user's emotions and personalizes the shopping experience.

[1247] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1248] Step 1:

[1249] A user accesses a virtual store using a terminal and enters authentication information (user name and password) into a login form. The terminal sends this authentication information to the server.

[1250] Input: Username and Password

[1251] Output: Sends authentication information to the server

[1252] What happens: The user enters their authentication information into a login form, and the device emails that information to the server.

[1253] Step 2:

[1254] The server verifies the user by checking the received authentication information against the metaverse database. If authentication is successful, the server generates a new session ID and sends it to the device.

[1255] Input: Credentials

[1256] Output: Session ID

[1257] Specific operation: The server performs a matching query against the database, and if authentication is successful, generates a session ID and sends it back to the device.

[1258] Step 3:

[1259] The terminal requests scene data and character data of the virtual store based on the session ID received from the server.

[1260] Input: Session ID

[1261] Output: Request for scene and character data

[1262] Specific operation: The terminal sends a request including the session ID to the server, requesting the necessary data.

[1263] Step 4:

[1264] The server retrieves scene data and character data from the metaverse database based on the session ID and sends them to the terminal.

[1265] Input: Session ID

[1266] Output: Scene data and character data

[1267] Specific operation: The server performs a database search, obtains scene data and character data, and sends them to the terminal.

[1268] Step 5:

[1269] The device renders a 3D environment of the virtual store based on the scene data and character data received and displays it to the user.

[1270] Input: Scene data and character data

[1271] Output: 3D environment of a virtual store

[1272] Specific operation: The device starts the rendering engine, draws the 3D environment based on the received data, and displays it on the screen.

[1273] Step 6:

[1274] The camera on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes the facial expressions to recognize the user's emotions. The recognized emotion information is then sent to the server.

[1275] Input: facial expression data

[1276] Output: Emotional information

[1277] Specific operation: The camera captures facial expressions, the emotion engine analyzes the facial expressions to recognize the emotions, and sends the information to the server.

[1278] Step 7:

[1279] The server dynamically suggests products suitable for the user based on the received emotion information, and sends the suggestions to the terminal.

[1280] Input: Emotion information

[1281] Output: Product proposal information

[1282] Specific operation: The server generates suggestions from a product database based on the emotion information and sends them to the device.

[1283] Step 8:

[1284] The terminal displays the product suggestion information received from the server to the user, and the user can view or purchase products according to the suggestion.

[1285] Input: Product proposal information

[1286] Output: Product display

[1287] Specific operation: The device displays the suggested information on the screen, and the user can browse or purchase.

[1288] The program and prompt to use

[1289] The programs used include "virtual_shopping_platform" and "emotion_recognition".

[1290] Example prompt sentence:

[1291] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[1292] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1293] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1294] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1295] [Third embodiment]

[1296] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1297] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1298] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1299] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1300] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1301] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1302] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1303] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1304] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1305] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1306] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1307] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1308] Basic system overview

[1309] The present invention is a system in which a user logs into the Metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. The operation of the system will be described in detail below for each processing step.

[1310] Overall system configuration

[1311] The system consists of the following main components:

[1312] 1. User Device

[1313] 2. Server System

[1314] 3. Metaverse Database

[1315] Login process

[1316] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[1317] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1318] Loading the Metaverse

[1319] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[1320] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[1321] Presentation of the game objective

[1322] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[1323] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1324] Cooperation and information gathering

[1325] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1326] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1327] Event Progression

[1328] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[1329] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[1330] Achieving goals and distributing rewards

[1331] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[1332] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[1333] The above is a specific description of the embodiment for carrying out the present invention.

[1334] The processing flow will be explained below.

[1335] Program processing steps

[1336] Step 1:

[1337] User Login

[1338] The user opens the login form using the terminal.

[1339] The terminal presents the user with a form for entering a username and password.

[1340] The user enters their authentication information and clicks the Login button.

[1341] The device sends the authentication information to the server.

[1342] The server checks the authentication information against a database and determines whether the authentication is successful.

[1343] If the server is successful in the authentication, it generates a session ID and sends it to the terminal.

[1344] The terminal displays a successful login message.

[1345] Step 2:

[1346] Loading the Metaverse

[1347] The terminal sends a request for scene data and character data to the server.

[1348] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[1349] The server acquires the character data and sends it to the device.

[1350] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[1351] Step 3:

[1352] Presentation of the game objective

[1353] The terminal requests game objective information from the server.

[1354] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[1355] The terminal displays the acquired game objective to the user.

[1356] Step 4:

[1357] Cooperation and information gathering

[1358] Users approach and interact with characters in the Metaverse.

[1359] The user inputs a question and sends it to the character.

[1360] The terminal transmits the query information to the server.

[1361] The server uses the character's AI engine to generate a response and send it to the device.

[1362] The terminal displays the response to the user.

[1363] Step 5:

[1364] Event Progression

[1365] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[1366] The server receives the user's behavior information and checks the event occurrence conditions.

[1367] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[1368] The device displays event information to the user and prompts them to take the next action.

[1369] Step 6:

[1370] Achieving goals and distributing rewards

[1371] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[1372] The server verifies the achievement of the objectives and generates reward information.

[1373] The server transmits the reward information to the terminal.

[1374] The terminal displays reward information to the user and notifies them that the game session is ending.

[1375] Specific example explanation

[1376] As a concrete example, we will explain the specific actions that "User A" takes from logging in to achieving his or her goal.

[1377] Step 1:

[1378] "User A" uses a terminal to open the login form page, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which checks the authentication information in the database. If the authentication is successful, the server generates a session ID and sends it back to the terminal. "User A"'s terminal displays a login success message.

[1379] Step 2:

[1380] After "User A" successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database and sends it to "User A's" device. The device renders the metaverse based on the received data and displays it to "User A."

[1381] Step 3:

[1382] "User A" operates the device to confirm the objective of the game, and the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays the information to "User A" and presents the objective of "find the lost treasure."

[1383] Step 4:

[1384] "User A" approaches a villager character (CP) and asks "Where is the treasure?" The terminal of "User A" sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the terminal. The terminal displays the response to "User A."

[1385] Step 5:

[1386] When "User A" heads towards the castle, the device sends this action information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates event information that "the castle gates have been magically closed" and sends it to the device. The device then displays this event information to "User A" and prompts him to take the next action.

[1387] Step 6:

[1388] "User A" collects the necessary items and finds the treasure. The device sends the information to the server, which verifies that the objective was achieved. The server generates reward information (e.g., in-game currency) and sends it to the device. The device displays the reward information to "User A" and notifies him or her that the game session is ending.

[1389] Example 1

[1390] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] Current virtual reality systems require complex system processing after a user logs in, which can degrade the quality of the user experience. Furthermore, when users interact with characters in the virtual space, conventional systems only provide limited responses, resulting in a lack of completeness as a comprehensive gaming environment. Furthermore, there are many instances where manual processing is required for reward distribution when a user achieves a goal, and automation is needed.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1393] In this invention, the server includes means for receiving and verifying authentication information for a user to log in to a virtual space, means for generating a session ID for the authenticated user, and means for generating question and answer prompts for a character in the virtual space using a generative AI model. This simplifies system processing after a user logs in, improving the quality of the user experience. It also enables more diverse and realistic responses in character interactions in the virtual space, improving the overall game environment. Furthermore, it automates reward allocation upon goal achievement, increasing overall efficiency.

[1394] "Authentication information" refers to a combination of a username and password that a user uses to log in to a virtual space.

[1395] A "server" is a central computer system for the entire virtual space system, providing data storage, processing, user authentication, and other functions.

[1396] A "session ID" is a unique identifier generated by the server to identify a user after the user has been authenticated.

[1397] A "terminal" is a device used by a user to access a virtual space, and includes a PC, smartphone, tablet, etc.

[1398] "Scene data" is data that constitutes the environment, background, objects, etc. within a virtual space.

[1399] "Character data" refers to data that includes information, attributes, actions, etc. of characters in a virtual space.

[1400] "Rendering" is the process of visually displaying a virtual 3D environment based on scene data and character data.

[1401] The "objective of the game" is the goal or task that the user must achieve within the virtual space.

[1402] A "generative AI model" is an algorithm or model that uses artificial intelligence technology to generate appropriate responses or content based on user input.

[1403] "Behavioral information" is data on the actions and choices made by the user within the virtual space.

[1404] An "event occurrence condition" is a condition or trigger element necessary for a specific event to occur in virtual space.

[1405] "Event information" is information that is generated when an event occurrence condition is met, and includes details of the event and the next action to be taken.

[1406] "Reward information" is details of the rewards a user will receive when they achieve a goal in the game. Examples include virtual currency, items, etc.

[1407] The present invention is a system in which a user logs into a virtual space and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Each component and its operation will be described in detail below.

[1408] User terminal

[1409] User devices include PCs, smartphones, tablets, and other devices. Users use these devices to access the virtual space and perform everything from login to gameplay. The devices use software such as HTML, JavaScript, React, and Three.js to execute the user interface and 3D rendering.

[1410] Server System

[1411] The server system is a central computer system that handles user authentication, database management, response generation using generative AI models, event management, etc. The server runs on software such as Node.js and MySQL.

[1412] Metaverse Database

[1413] The metaverse database stores user information, scene data, character data, game objectives, event information, etc. MySQL is used as the database management system.

[1414] Login process

[1415] The user displays a login form on their device and enters their username and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a new session ID and returns it to the device. The device notifies the user of successful authentication and displays an entry point to the virtual space.

[1416] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1417] Loading the Metaverse

[1418] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders a 3D virtual environment based on the received data and displays it to the user.

[1419] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the virtual space based on that data.

[1420] Presentation of the game objective

[1421] The device requests game objective information from the server. The server retrieves the objective information from a database based on the user's session ID. The server sends the objective information to the device, which displays it to the user.

[1422] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1423] Cooperation and information gathering

[1424] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine (generative AI model) to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1425] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1426] Prompt Sentence Examples

[1427] User prompt: "Where is the treasure?"

[1428] Event Progression

[1429] When a user takes a specific action (e.g., heads to the castle), the device sends the action information to the server. The server checks the event occurrence conditions and generates event information if the conditions are met. The server then sends the event information to the device, which then displays it to the user.

[1430] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[1431] Achieving goals and distributing rewards

[1432] When the user achieves the game objective, the device sends that information to the server. The server confirms the achievement and generates reward information (e.g., in-game currency). The server then sends the reward information to the device, which displays it to the user. This ends the game session.

[1433] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information, and sends it to the device. The device displays it to "User A" and notifies him that the game is over.

[1434] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1435] Step 1:

[1436] The user enters their username and password into the login form on their device.

[1437] Input: Username, Password

[1438] Specific behavior: The device displays the login screen using HTML and JavaScript. The user fills out the form and clicks the login button.

[1439] Step 2:

[1440] The terminal transmits the entered authentication information to the server.

[1441] Input: Username, Password

[1442] Specific operation: Sends authentication information to the server via an HTTP POST request using the JavaScript Fetch API.

[1443] Step 3:

[1444] The server checks the credentials against a database.

[1445] Input: Username, Password

[1446] Output: Authentication success or failure, session ID

[1447] What happens: The server uses Node.js to query the MySQL database for authentication information. If the credentials match, it generates a new session ID with a UUID.

[1448] Step 4:

[1449] The server returns the authentication result to the terminal and includes the new session ID.

[1450] Input: Authentication success or failure, session ID

[1451] Output: Session ID

[1452] Specific operation: If authentication is successful, the session ID is returned to the terminal in the HTTP response.

[1453] Step 5:

[1454] The terminal notifies the user of successful authentication and displays an entry point to the virtual space.

[1455] Input: Session ID

[1456] Specific behavior: The device displays a successful authentication message and an entry point link. The screen is updated using React.

[1457] Step 6:

[1458] The user clicks on an entry point into the virtual space.

[1459] Input: Entry point URL, Session ID

[1460] Specific operation: The terminal sends a request to the entry point to the server.

[1461] Step 7:

[1462] The terminal sends a request for scene data and character data to the server.

[1463] Input: Session ID

[1464] Specific operation: The device requests scene data and character data from the server using an HTTP GET request.

[1465] Step 8:

[1466] The server retrieves scene data and character data from the database and transmits them to the terminal.

[1467] Input: Session ID

[1468] Output: Scene data, character data

[1469] Specific operation: The server uses Node.js to retrieve the necessary data from the MySQL database and send it to the terminal as an HTTP response.

[1470] Step 9:

[1471] The device renders a 3D virtual environment based on the scene data and character data it receives and displays it to the user.

[1472] Input: Scene data, character data

[1473] What it does: The device uses Three.js to render the scene and characters in 3D and display them in the user interface.

[1474] Step 10:

[1475] The terminal requests information for game purposes from the server.

[1476] Input: Session ID

[1477] Specific operation: The device requests the game objective from the server via an HTTP GET request.

[1478] Step 11:

[1479] The server retrieves game objective information from the database based on the user's session ID and sends it to the terminal.

[1480] Input: Session ID

[1481] Output: Game objective information

[1482] Specific operation: The server uses Node.js to retrieve game objective information from the MySQL database and sends it to the terminal as an HTTP response.

[1483] Step 12:

[1484] The terminal displays the received game objective information to the user.

[1485] Input: Game objective information

[1486] Specific operation: The device uses React to display game objective information on the user interface.

[1487] Step 13:

[1488] The user sends a question to a character in the virtual space.

[1489] Input: User's question

[1490] Specific operation: The device sends the question to the server.

[1491] Step 14:

[1492] The terminal sends the user's question to the server.

[1493] Input: User's question

[1494] Specific operation: The question is sent to the server via an HTTP POST request, possibly using WebSocket.

[1495] Step 15:

[1496] The server uses an AI engine (generative AI model) to generate a response to the question and send it to the terminal.

[1497] Input: User's question

[1498] Output: Response by AI engine

[1499] Specific operation: The server uses a generative AI model (e.g., GPT-3) to generate an answer to the question and sends it to the device in an HTTP response.

[1500] Step 16:

[1501] The terminal displays the received response to the user.

[1502] Input: Response by AI engine

[1503] Specific behavior: The device displays the response in the user interface using React.

[1504] Step 17:

[1505] The user takes a specific action (e.g., heads to the castle).

[1506] Input: User behavior information

[1507] Specific operation: The device sends its behavior information to the server.

[1508] Step 18:

[1509] The terminal transmits the user's behavior information to the server.

[1510] Input: User behavior information

[1511] Specific operation: Sends behavioral information to the server via an HTTP POST request.

[1512] Step 19:

[1513] The server checks the event occurrence conditions, and if the conditions are met, generates event information and sends it to the terminal.

[1514] Input: User behavior information

[1515] Output: Event information

[1516] Specific operation: The server checks the event occurrence conditions based on the behavioral information, and if the conditions are met, it generates event information using Node.js and sends it to the terminal as an HTTP response.

[1517] Step 20:

[1518] The terminal displays the received event information to the user.

[1519] Input: Event information

[1520] Specific behavior: The device displays the event information on the user interface using React.

[1521] Step 21:

[1522] The user completes the game objective (e.g., finds treasure).

[1523] Input: Objective achievement information

[1524] Specific operation: The device sends the information to the server.

[1525] Step 22:

[1526] The terminal transmits the goal achievement information to the server.

[1527] Input: Objective achievement information

[1528] Specific behavior: Sends achievement information to the server using an HTTP POST request.

[1529] Step 23:

[1530] The server verifies the achievement of the objectives and generates reward information.

[1531] Input: Objective achievement information

[1532] Output: Reward information

[1533] Specific operation: The server confirms that the goal has been achieved and generates reward information using Node.js.

[1534] Step 24:

[1535] The server transmits the reward information to the terminal.

[1536] Input: Reward information

[1537] Specific operation: Reward information is sent to the terminal in an HTTP response.

[1538] Step 25:

[1539] The terminal displays the received reward information to the user.

[1540] Input: Reward information

[1541] Specific behavior: The device displays the reward information on the user interface using React and notifies the end of the game session.

[1542] (Application example 1)

[1543] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1544] There is a need for an effective method to facilitate interactions with characters and event progression in order for users to achieve specific goals in the metaverse. There is also a need to make the shopping experience more attractive by allowing users to receive special benefits such as coupons when purchasing specific products in virtual stores.

[1545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1546] In this invention, the server includes: means for sending authentication information for a user to log in to the metaverse; means for the server to receive the authentication information and authenticate the user; means for the server to generate a session ID for the authenticated user; means for the terminal to request scene data and character data from the server; means for the server to acquire the scene data and character data and send it to the terminal; means for the terminal to render the metaverse based on the scene data and character data received; means for the terminal to request a game objective from the server and display the received objective; means for the user to send a question to a character in the metaverse; means for the server to generate a response to the question and send it to the terminal; means for the terminal to display the response received by the user; means for sending user behavior information to the server and confirming the event occurrence conditions; means for the server to generate event information and send it to the terminal if the conditions are met; means for the server to display the event information received by the terminal to the user; means for the server to verify the user's achievement of the objective and generate reward information; means for sending the reward information to the terminal and displaying it; means for generating a coupon when the user purchases a specific product and sending it to the terminal; and means for receiving a prompt message regarding the purchase of a specific product using a generative AI model. This allows users to smoothly interact with and progress through events within the metaverse, and also makes it possible to offer coupons when purchasing certain products.

[1547] The "metaverse" is a three-dimensional virtual environment in which users can interact with other users and virtual characters within a virtual space.

[1548] "Authentication information" refers to information such as a username and password that a user uses when accessing the metaverse.

[1549] A "session ID" is a unique identification number generated by the server for an authenticated user.

[1550] "Scene data" is data about the environment and objects within the metaverse.

[1551] "Character data" refers to data about characters and NPCs (Non-Player Characters) controlled by users within the Metaverse.

[1552] "Rendering" is the process by which a computer generates a visually displayable image from three-dimensional scene data.

[1553] A "coupon" is electronic information indicating a discount or benefit that is offered when a user meets certain conditions.

[1554] A "prompt sentence" is textual input information that a generative AI model uses to generate a response.

[1555] A "generative AI model" is an artificial intelligence model that generates an appropriate response based on a given prompt sentence.

[1556] "Event information" is information about specific actions or events that occur within the metaverse in response to user behavior conditions.

[1557] "Reward information" is information regarding rewards that will be provided to users when they achieve specific goals within the metaverse.

[1558] A "virtual store" is a store that exists virtually on the Internet, and is a virtual space where users can gather product information and make purchases.

[1559] Basic system overview

[1560] The present invention is a system that allows users to log in to a virtual store and collect product information to achieve specific goals, or to receive rewards such as coupons as they progress through events. The operation of the system is described in detail below.

[1561] Overall system configuration

[1562] The system mainly consists of the following major components:

[1563] 1. User device: Smartphone application, etc.

[1564] 2. Server system: A server that processes and manages data.

[1565] 3. Database: A database that manages metaverse scene data, character data, product data, etc.

[1566] 4. Generative AI model: An artificial intelligence model that generates appropriate responses or coupons based on user prompts.

[1567] Login process

[1568] The user accesses the virtual store using their terminal. A login form is displayed, the user enters their authentication information (username and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database. If authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authorized and the entry point to the virtual store is displayed.

[1569] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends "User A's" authentication information to the server, which then verifies it in the database, generates a session ID, and sends it to "User A's" terminal.

[1570] Loading a Virtual Store

[1571] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the terminal. The terminal then renders a 3D environment of the virtual store based on this data and displays it to the user.

[1572] As a concrete example, the terminal of "User A" requests scene data and product data from the server, and the server retrieves this data from the database and sends it to the terminal of "User A." The terminal then displays a virtual store based on that data.

[1573] State your shopping purpose

[1574] The terminal requests shopping purpose information from the server, and the server obtains the purpose information based on the user's session ID and sends it to the terminal, which then displays it to the user, allowing them to take the first step toward achieving their purpose.

[1575] For example, "User A" sends a request to the server to confirm his / her shopping purpose on his / her device, and the server acquires the purpose information "to find discount sales" and sends it to "User A's" device. The device displays it, allowing "User A" to recognize his / her purpose.

[1576] Cooperation and information gathering

[1577] Users can interact with store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question, the device sends the question to the server. The server uses a generative AI model to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1578] For example, "User A" asks a store clerk character for "details of the most popular item." The device sends the question to the server, and the server generates a response saying, "This item is currently on sale, 20% off," and sends it to the device. The device displays it to "User A."

[1579] Event Progression

[1580] When a user purchases a specific product, the behavioral information is sent from the device to the server. The server checks the purchase conditions, and if the conditions are met, the server generates event information and sends it to the device. The device displays the event information to the user, prompting them to take the next action.

[1581] For example, when "User A" places a specific product in their cart and purchases it, their device sends that information to the server. The server confirms that the conditions are met, generates event information for a "10% off coupon for your next purchase," and sends it to their device. The device then displays this to "User A."

[1582] Achieving goals and distributing rewards

[1583] When the user achieves their shopping goal, the terminal sends the information to the server. The server verifies the goal achievement and generates reward information and sends it to the terminal. The terminal displays the reward information to the user, and the shopping session ends.

[1584] As a concrete example, when "User A" achieves his / her goal in a virtual store, the device sends this information to the server. The server confirms that the goal has been achieved, generates a "coupon for next use" and sends it to the device. The device displays this to "User A" and notifies him / her that his / her shopping has finished.

[1585] An example prompt is:

[1586] "Generate a 10% off coupon for the next purchase by the user when they purchase a specific product."

[1587] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1588] Step 1:

[1589] A user accesses a virtual store from a terminal, enters authentication information (user name and password) in the login form, and presses the login button. The entered authentication information is sent from the terminal to the server. The server compares the received authentication information against a database and performs authentication. If authentication is successful, the server generates a new session ID and sends it back to the terminal. The terminal then notifies the user that access to the system has been permitted, and displays the entry point to the virtual store.

[1590] Input: Authentication information (username, password)

[1591] Output: Session ID

[1592] Step 2:

[1593] After the user successfully logs in, the terminal sends a request for scene data and product data to the server. The server retrieves the metaverse scene data and product data from the database based on the session ID and sends them to the terminal. The terminal then renders the 3D environment of the virtual store based on this data and displays it to the user.

[1594] Input: Session ID

[1595] Output: Scene data and product data

[1596] Step 3:

[1597] The terminal sends a request for shopping intent information to the server. The server retrieves the intent information from the database based on the session ID and sends it to the terminal. The terminal displays the intent information to the user and prompts them to take their first action.

[1598] Input: Session ID

[1599] Output: Shopping intent information

[1600] Step 4:

[1601] Users can ask questions to store clerks and other characters in the virtual store. When a user approaches a specific character and asks a question such as "Details of the most popular item," the device sends the question to the server. The server uses a generative AI model to generate an appropriate response to the question and sends it to the device. The device then displays the received response to the user.

[1602] Input: User prompt

[1603] Output: Response

[1604] Step 5:

[1605] When a user places a specific product in their cart and purchases it, that behavioral information is sent from the device to the server. The server checks the purchase conditions based on this information, and if the conditions are met, generates event information and sends it to the device. For example, if the conditions are met, the server generates a "10% off coupon for your next purchase" and sends it to the device. The device then displays this coupon information to the user.

[1606] Input: Purchase Information

[1607] Output: Coupon information

[1608] Step 6:

[1609] When the user achieves their shopping goal, the terminal sends the information to the server, which verifies the goal achievement, generates reward information, and sends it to the terminal, which displays the reward information to the user and notifies them that the shopping session has ended.

[1610] Input: Objective achievement information

[1611] Output: Reward information

[1612] The above are the specific processing steps of the system of the present invention. This allows users to smoothly interact with the virtual store and progress through events. Furthermore, coupons can be obtained when purchasing specific products, which is expected to improve the shopping experience.

[1613] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1614] Basic system overview

[1615] The present invention is a system in which a user logs into the metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience within the metaverse based on the user's emotions.

[1616] Overall system configuration

[1617] The system consists of the following main components:

[1618] 1. User Device

[1619] 2. Server System

[1620] 3. Metaverse Database

[1621] 4. Emotion Engine

[1622] Login process

[1623] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[1624] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1625] Loading the Metaverse

[1626] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[1627] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[1628] Presentation of the game objective

[1629] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[1630] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1631] Cooperation and information gathering

[1632] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1633] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1634] Event Progression

[1635] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[1636] As a concrete example, when "User A" heads to the castle, the device sends this behavioral information to the server. The server confirms that the conditions are met, generates event information indicating that the castle gate has been magically closed, and sends it to the device. The device then displays this event information to "User A."

[1637] Achieving goals and distributing rewards

[1638] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[1639] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[1640] Incorporating an emotion engine

[1641] The emotion engine analyzes the user's facial expressions and voice to recognize their emotions, which are then sent to the server, which then dynamically adjusts events and interactions within the metaverse.

[1642] As a concrete example, when "User A" talks to a villager character in the metaverse, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays it to "User A."

[1643] The introduction of the emotion engine personalizes the gaming experience based on the real-time emotions felt by the user, providing an even more immersive experience.

[1644] The processing flow will be explained below.

[1645] Program processing steps

[1646] Step 1:

[1647] User Login

[1648] The user opens the login form using the terminal.

[1649] The terminal presents the user with a form for entering a username and password.

[1650] The user enters their authentication information and clicks the Login button.

[1651] The device sends the authentication information to the server.

[1652] The server checks the authentication information against a database and determines whether the authentication is successful.

[1653] If the authentication is successful, the server generates a session ID and sends it to the terminal.

[1654] The terminal displays a successful login message.

[1655] Step 2:

[1656] Loading the Metaverse

[1657] The terminal sends a request for scene data and character data to the server.

[1658] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[1659] The server also obtains character data and sends it to the device.

[1660] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[1661] Step 3:

[1662] Early recognition of emotions

[1663] When a user logs into the metaverse on their device, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotions.

[1664] The emotion engine transmits the recognized emotion information to the server.

[1665] The server stores the emotional information in a database and uses it in subsequent gameplay.

[1666] Step 4:

[1667] Presentation of the game objective

[1668] The terminal requests game objective information from the server.

[1669] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[1670] The terminal displays the acquired game objective to the user.

[1671] Step 5:

[1672] Cooperation and information gathering

[1673] Users approach and interact with characters in the Metaverse.

[1674] The user inputs a question and sends it to the character.

[1675] The terminal transmits the query information to the server.

[1676] The server uses the character's AI engine to generate a response and send it to the device.

[1677] The terminal displays the response to the user.

[1678] Step 6:

[1679] Emotional remonitoring

[1680] The emotion engine periodically analyzes the user's facial expressions and voice, recognizing emotions in real time.

[1681] The recognized emotion information is sent back to the server.

[1682] The server dynamically adjusts events and character responses based on the emotional information.

[1683] Step 7:

[1684] Event Progression

[1685] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[1686] The server receives the user's behavior information and checks the event occurrence conditions.

[1687] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[1688] The device displays event information to the user and prompts them to take the next action.

[1689] Step 8:

[1690] Achieving goals and distributing rewards

[1691] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[1692] The server verifies the achievement of the objectives and generates reward information.

[1693] The server transmits the reward information to the terminal.

[1694] The terminal displays reward information to the user and notifies them that the game session is ending.

[1695] Examples of emotion engines

[1696] As a concrete example, we will explain the process by which "User A" logs in to the Metaverse and plays.

[1697] Step 1:

[1698] "User A" opens the login form on the terminal, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which verifies it against the database. If authentication is successful, the server sends a session ID, and the terminal displays a login success message.

[1699] Step 2:

[1700] After "User A" logs in, the device sends a request for scene and character data to the server. The server retrieves the data and sends it to the device. The device renders the metaverse and displays it to the user.

[1701] Step 3:

[1702] When "User A" enters the metaverse, the device's built-in emotion engine analyzes the user's facial expressions and voice to recognize their emotions and transmits that information to the server, which stores the emotion information and uses it for subsequent processing.

[1703] Step 4:

[1704] In order for "User A" to confirm the objective of the game, the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays it to "User A" and presents the objective of "find the treasure."

[1705] Step 5:

[1706] "User A" asks a villager character "Where is the treasure?" The device sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the device. The device displays it to "User A."

[1707] Step 6:

[1708] While "User A" is interacting with a villager, the emotion engine recognizes tension from "User A's" facial expression. The server receives this information and adjusts the villager character's response to encourage relaxation. The device then converts the villager character's response into a suggestion for "what to do to relax" and displays it to the user.

[1709] Step 7:

[1710] When "User A" heads to the castle, the device sends that behavioral information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates new event information stating that "the castle gates have been magically closed" and sends it to the device. The device displays this to "User A" and prompts them to take the next action.

[1711] Step 8:

[1712] When "User A" finds the treasure, the device sends the achievement information to the server. The server confirms the achievement, generates reward information (e.g., in-game currency), and sends it to the device. The device displays the information to "User A" and notifies him or her that the game session is ending.

[1713] Example 2

[1714] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1715] Conventional virtual reality systems lack the ability to recognize users' emotions in real time and dynamically adjust the experience based on those emotions, which results in a lack of immersion and satisfaction for users.

[1716] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a session ID for an authenticated user, a means for acquiring scene data and character data and transmitting them to the terminal, and a means for transmitting user behavior information to the server and checking the event occurrence conditions. This allows the user's emotions to be recognized in real time after logging in, and the experience in the virtual space to be dynamically adjusted.

[1717] "Authentication information" is data such as a username and password that a user uses to access a system.

[1718] A "session ID" is a unique identifier generated by the server to identify operations by an authenticated user.

[1719] "Scene data" is data that includes information about the environment and objects in a virtual space.

[1720] "Character data" is data that includes information about characters in a virtual space.

[1721] A "virtual space" is a computer-generated 3D environment that can be accessed and experienced by a user.

[1722] "Rendering" is the process by which a computer generates a graphical display based on scene data and character data.

[1723] "Game objective" refers to the tasks or goals that a user must accomplish within the virtual space.

[1724] "Event information" is information about an event that occurs when certain conditions are met in the virtual space.

[1725] "Reward information" is information about the reward that a user can receive when they achieve the objective of the game.

[1726] An "emotion engine" is software that recognizes emotions in real time from a user's facial expressions, voice, etc.

[1727] A "virtual character" is a computer-generated character that interacts with a user in a virtual space.

[1728] "Interaction" refers to the actions or operations that a user performs to interact with elements in a virtual space.

[1729] Basic system overview

[1730] The present invention is a system in which a user logs into a virtual space and cooperates with other characters to gather information and progress through events in order to achieve a specific goal. Furthermore, it incorporates an emotion engine that recognizes the user's emotions, dynamically adjusting the experience in the virtual space based on the user's emotions.

[1731] Overall system configuration

[1732] The system consists of the following main components:

[1733] 1. User Device

[1734] 2. Server System

[1735] 3. Virtual Space Database

[1736] 4. Emotion Engine

[1737] Hardware and software used

[1738] User device: Computer or smartphone (e.g., Windows PC, smartphone)

[1739] Server systems: high-performance servers (e.g. cloud infrastructure)

[1740] Virtual spatial database: Data management system (e.g., MySQL, NoSQL database)

[1741] Emotion engine: Emotion recognition software (e.g., Emotion SDK)

[1742] Login process

[1743] A user accesses the virtual space system using their own terminal. A login form is displayed, and the user enters authentication information (user name and password) and clicks the login button. The terminal sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the terminal. The terminal notifies the user that access to the system has been authenticated, and the entry point to the virtual space is displayed. As a concrete example, "User A" attempts to log in on the terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1744] Loading the virtual world

[1745] When a user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders and displays a 3D environment in the virtual space based on this data. As a concrete example, after "User A" has successfully logged in, the device requests scene data and character information from the server. The server retrieves this data and sends it to "User A's" device, and the device then displays the virtual space based on that data.

[1746] Presentation of the game objective

[1747] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays this to the user, allowing them to take the first step towards achieving their objective. As a concrete example, when "User A" wants to confirm the game objective, the device sends a request to the server. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1748] Cooperation and information gathering

[1749] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device displays the response to the user. As a concrete example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1750] Event Progression

[1751] When a user takes a specific action (for example, heading towards the castle), the device sends that action information to the server, and the server checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device displays that event information to the user and prompts them to take the next action. As a specific example, when "User A" heads towards the castle, the device sends that action information to the server. The server checks that the conditions are met, generates event information that the castle gates have been magically closed, and sends it to the device. The device displays that event information to "User A."

[1752] Achieving goals and distributing rewards

[1753] When a user achieves the game objective, the device sends that information to the server. The server verifies that the objective has been achieved, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends. As a specific example, when "User A" finds treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays it to "User A," notifying him that the game is over.

[1754] Incorporating an emotion engine

[1755] The emotion engine recognizes emotions by analyzing the user's facial expressions and voice. The recognized emotional information is sent to the server, which then dynamically adjusts events and interactions in the virtual space based on that information. As a specific example, when "User A" talks to a villager character in the virtual space, the emotion engine recognizes stress from "User A's" facial expression. The server receives the recognized stress information, generates a response from the villager character suggesting "what to do to relax," and sends it to the device. The device displays this to "User A." The introduction of the emotion engine personalizes the game experience based on the user's real-time emotions, providing an even more immersive experience.

[1756] Prompt Sentence Examples

[1757] "User A logs into a virtual world and works with his friends to find treasure. On the way to the castle, the emotion engine recognizes the user's stress, and a villager character suggests ways for the user to relax. Please explain this scenario in natural language."

[1758] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1759] Step 1: Login process

[1760] 1. Input: Authentication information (user name, password)

[1761] 2. Specific operation:

[1762] The user enters their username and password into the login form.

[1763] The terminal transmits the entered authentication information to the server.

[1764] 3. Data processing and calculation:

[1765] The server checks the received authentication information against its database.

[1766] A database query is performed to verify that the username and password match.

[1767] 4. Output: Authentication result, session ID (if successful)

[1768] 5. Specific operation:

[1769] If the authentication is successful, the server generates a new session ID and sends it to the terminal.

[1770] The terminal receives the session ID and notifies the user that the login was successful.

[1771] Step 2: Loading the Virtual Space

[1772] 1. Input: Session ID

[1773] 2. Specific operation:

[1774] After the user has successfully logged in, the terminal sends a request for scene data and character data to the server.

[1775] 3. Data processing and calculation:

[1776] The server retrieves scene data and character data from the database based on the session ID.

[1777] Execute database queries and extract the required data.

[1778] 4. Output: Scene data, character data

[1779] 5. Specific operation:

[1780] The server transmits the acquired data to the terminal.

[1781] The device renders a 3D environment based on the data it receives and displays the virtual space.

[1782] Step 3: Present the objective of the game

[1783] 1. Input: Session ID

[1784] 2. Specific operation:

[1785] The terminal requests game objective information from the server.

[1786] 3. Data processing and calculation:

[1787] The server retrieves the desired information from the database based on the session ID.

[1788] Execute a database query to extract the desired information.

[1789] 4. Output: Game objective information

[1790] 5. Specific operation:

[1791] The server transmits the target information to the terminal.

[1792] The terminal displays the received objective information to the user and guides the user to the next step.

[1793] Step 4: Collaborate and gather information

[1794] 1. Input: User's question

[1795] 2. Specific operation:

[1796] The user inputs a question to the character and sends it.

[1797] The device sends the question to the server.

[1798] 3. Data processing and calculation:

[1799] The server uses a generative AI model to generate answers to questions.

[1800] Feed the AI ​​model with questions and generate appropriate responses.

[1801] 4. Output: Response from the character

[1802] 5. Specific operation:

[1803] The server generates a response and sends it to the terminal.

[1804] The terminal displays the response to the user.

[1805] Step 5: Event Progression

[1806] 1. Input: User behavior information

[1807] 2. Specific operation:

[1808] The user takes a specific action (e.g., heads to the castle).

[1809] The terminal transmits the behavior information to the server.

[1810] 3. Data processing and calculation:

[1811] The server executes a database query to check the conditions under which the event occurs.

[1812] If the condition is met, new event information is generated.

[1813] 4. Output: Event information

[1814] 5. Specific operation:

[1815] The server transmits the event information to the terminal.

[1816] The device displays the event information to the user and guides them on the next action.

[1817] Step 6: Achieving Objectives and Distributing Rewards

[1818] 1. Input: Objective achievement information

[1819] 2. Specific operation:

[1820] When the user achieves the goal, the terminal transmits the information to the server.

[1821] 3. Data processing and calculation:

[1822] The server verifies the achievement information and checks it against a database.

[1823] After confirming that the objective has been achieved, reward information is generated.

[1824] 4. Output: Reward information

[1825] 5. Specific operation:

[1826] The server transmits the generated reward information to the terminal.

[1827] The terminal displays reward information to the user and notifies the user of the end of the game session.

[1828] Step 7: Incorporating the Emotion Engine

[1829] 1. Input: User's facial expression data, voice data

[1830] 2. Specific operation:

[1831] The emotion engine analyzes the user's facial expressions and voice in real time.

[1832] 3. Data processing and calculation:

[1833] The emotion engine generates the recognized emotion information and sends it to the server.

[1834] The server dynamically adjusts events and interactions based on the emotional information.

[1835] 4. Output: Coordinated event information, interaction information

[1836] 5. Specific operation:

[1837] The server transmits the adjusted event information to the terminal.

[1838] The device displays tailored event information to the user, personalizing the experience in the virtual space.

[1839] (Application example 2)

[1840] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1841] In current virtual stores, the experience is not fully personalized based on the user's emotions. This makes it difficult for users to smoothly browse the store and receive optimal product recommendations without feeling anxious or confused. Furthermore, there is a demand for a more immersive and interactive experience for users of the metaverse.

[1842] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1843] In this invention, the server includes means for transmitting authentication information for a user to log in to the metaverse, means for the server to receive the authentication information and authenticate the user, means for the server to generate a session ID for the authenticated user, means for the terminal to request scene data and character data from the server, means for the server to acquire the scene data and character data and send it to the terminal, means for the terminal to render the metaverse based on the scene data and character data received, means for the terminal to request a game objective from the server and display the received objective, means for the user to send a question to a character in the metaverse, and means for the server to generate a response to the question and send it to the terminal. The device includes a means for receiving a response from the user, a means for displaying the response received by the terminal to the user, a means for transmitting user behavior information to a server and checking an event occurrence condition, a means for generating event information and transmitting it to the terminal if the condition is met by the server, a means for displaying the event information received by the terminal to the user, a means for the server to verify that the user has achieved their goal and generate reward information, a means for transmitting the reward information to the terminal and displaying it, a means for capturing the user's facial expression with a camera mounted on the terminal, a means for providing an emotion engine that recognizes the user's emotion from the captured facial expression, a means for dynamically adjusting the experience in the metaverse based on the recognized emotion, and a means for the terminal to suggest and notify products based on the user's emotion. This dynamically optimizes the experience in the virtual store based on the user's emotion, enabling a more immersive shopping experience.

[1844] The "metaverse" refers to a virtual space built on the Internet where users can interact with other users and the environment in real time via avatars.

[1845] "Authentication information" refers to information such as a username and password used when a user accesses the metaverse, and is data used to verify the user's identity.

[1846] A "session ID" is a unique identifier that the server generates for an authenticated user and is used to manage the user's session.

[1847] "Scene data" is data that makes up the environment within the metaverse, and includes information such as three-dimensional landscapes, objects, and backgrounds.

[1848] "Character data" refers to information about the avatars used by users and NPCs (non-player characters) within the Metaverse, including their appearance, movements, attributes, etc.

[1849] "Rendering" refers to the process by which a computer visually depicts a three-dimensional environment within the Metaverse based on scene and character data.

[1850] "Game Objective" is information about a specific task or mission that a user must accomplish within the Metaverse.

[1851] A "question" is information that a user inquires about or verifies with a character in the metaverse.

[1852] A "response" is a response that the server generates in response to a user's question and that is provided to the user as part of the interaction.

[1853] "Behavioral information" is data on the actions a user takes within the metaverse, and is information that is used to check whether an event will occur.

[1854] "Event information" is data about specific occurrences that occur within the metaverse and are triggered by user actions or conditions.

[1855] "Reward information" is reward data generated by the server when a user achieves a game objective, and is provided to the user.

[1856] An "emotion engine" refers to an algorithm or software that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[1857] "Dynamic adjustment" refers to the process of changing the experience within the metaverse in real time based on the user's current situation and emotions.

[1858] "Product suggestion" is the process of recommending appropriate products to users based on their emotions and behavior.

[1859] Basic system overview

[1860] The present invention is a system that utilizes an emotion engine to dynamically adjust a user's experience when they log in to a virtual store and engage in a purchasing experience. This system is composed of the following main components:

[1861] 1. User device: A device that accesses the metaverse and has a camera, such as smart glasses or a smartphone.

[1862] 2. Server system: Manages authentication information, provides scene data, generates questions and answers, and dynamically adjusts the experience.

[1863] 3. Metaverse Database: Stores scene data, character data, and product information within the Metaverse.

[1864] 4. Emotion Engine: Software that recognizes user emotions and dynamically adjusts the user experience based on that information.

[1865] Login process

[1866] The user accesses the virtual store using their own terminal and enters authentication information (user name and password) into the login form. The server receives this information and authenticates it by checking it against the metaverse database. If authentication is successful, the server generates a new session ID and returns it to the user's terminal. The terminal notifies the user that login was successful, and the entry point for the virtual store is displayed.

[1867] Loading a Virtual Store

[1868] After the user successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data based on the session ID and sends it to the device. The device then renders a 3D virtual store environment based on the received data and displays it to the user.

[1869] Emotion-based product recommendations

[1870] A camera installed on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes them to recognize the user's emotions. The recognized emotion information is sent to the server, which dynamically adjusts product suggestions and notifications based on that information. For example, if the user is excited, the server will suggest limited-time offers. Conversely, if the user is confused, the server will display a support message.

[1871] Specific examples

[1872] As users browse products in the virtual store, a camera captures their facial expressions, and an emotion engine recognizes "excitement." The server receives this data and sends information about limited-time offers to the user's device, which then displays the information to the user.

[1873] Example prompts for generative AI models

[1874] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[1875] Hardware and software used

[1876] Hardware: Smart glasses, smartphone, on-board camera

[1877] Software: OpenCV (used for acquiring and preprocessing camera images), emotion_recognition (used for emotion recognition), virtual_shopping_platform (virtual store API)

[1878] In this way, a system is realized that dynamically optimizes the experience in the virtual store based on the user's emotions and personalizes the shopping experience.

[1879] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1880] Step 1:

[1881] A user accesses a virtual store using a terminal and enters authentication information (user name and password) into a login form. The terminal sends this authentication information to the server.

[1882] Input: Username and Password

[1883] Output: Sends authentication information to the server

[1884] What happens: The user enters their authentication information into a login form, and the device emails that information to the server.

[1885] Step 2:

[1886] The server verifies the user by checking the received authentication information against the metaverse database. If authentication is successful, the server generates a new session ID and sends it to the device.

[1887] Input: Credentials

[1888] Output: Session ID

[1889] Specific operation: The server performs a matching query against the database, and if authentication is successful, generates a session ID and sends it back to the device.

[1890] Step 3:

[1891] The terminal requests scene data and character data of the virtual store based on the session ID received from the server.

[1892] Input: Session ID

[1893] Output: Request for scene and character data

[1894] Specific operation: The terminal sends a request including the session ID to the server, requesting the necessary data.

[1895] Step 4:

[1896] The server retrieves scene data and character data from the metaverse database based on the session ID and sends them to the terminal.

[1897] Input: Session ID

[1898] Output: Scene data and character data

[1899] Specific operation: The server performs a database search, obtains scene data and character data, and sends them to the terminal.

[1900] Step 5:

[1901] The device renders a 3D environment of the virtual store based on the scene data and character data received and displays it to the user.

[1902] Input: Scene data and character data

[1903] Output: 3D environment of a virtual store

[1904] Specific operation: The device starts the rendering engine, draws the 3D environment based on the received data, and displays it on the screen.

[1905] Step 6:

[1906] The camera on the user's device captures the user's facial expressions in real time, and the emotion engine analyzes the facial expressions to recognize the user's emotions. The recognized emotion information is then sent to the server.

[1907] Input: facial expression data

[1908] Output: Emotional information

[1909] Specific operation: The camera captures facial expressions, the emotion engine analyzes the facial expressions to recognize the emotions, and sends the information to the server.

[1910] Step 7:

[1911] The server dynamically suggests products suitable for the user based on the received emotion information, and sends the suggestions to the terminal.

[1912] Input: Emotion information

[1913] Output: Product proposal information

[1914] Specific operation: The server generates suggestions from a product database based on the emotion information and sends them to the device.

[1915] Step 8:

[1916] The terminal displays the product suggestion information received from the server to the user, and the user can view or purchase products according to the suggestion.

[1917] Input: Product proposal information

[1918] Output: Product display

[1919] Specific operation: The device displays the suggested information on the screen, and the user can browse or purchase.

[1920] The program and prompt to use

[1921] The programs used include "virtual_shopping_platform" and "emotion_recognition".

[1922] Example prompt sentence:

[1923] "When a user visits a virtual store and browses products, the emotion engine analyzes the user's facial expression and recognizes that it is 'happy.' What products should the AI ​​suggest to the user?"

[1924] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1925] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1926] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1927] [Fourth embodiment]

[1928] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1929] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1930] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1931] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1932] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1933] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1934] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1935] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1936] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1937] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1938] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1939] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1940] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1941] Basic system overview

[1942] The present invention is a system in which a user logs into the Metaverse and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. The operation of the system will be described in detail below for each processing step.

[1943] Overall system configuration

[1944] The system consists of the following main components:

[1945] 1. User Device

[1946] 2. Server System

[1947] 3. Metaverse Database

[1948] Login process

[1949] A user accesses the metaverse using their device. A login form is displayed, and the user enters their authentication information (username and password) and clicks the login button. The device sends this information to the server, which checks the authentication information against a database to determine whether the authentication was successful. If the authentication is successful, a new session ID is generated and sent back to the device. The device notifies the user that access to the system has been authorized, and the entry point to the metaverse is displayed.

[1950] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[1951] Loading the Metaverse

[1952] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then uses this data to render and display the 3D environment of the Metaverse.

[1953] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the metaverse based on that data.

[1954] Presentation of the game objective

[1955] The device requests the game's objective information from the server. The server obtains the objective information based on the user's session ID and sends it to the device. The device displays it to the user, allowing them to take the first step toward completing their objective.

[1956] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[1957] Cooperation and information gathering

[1958] Users can interact with other characters in the metaverse. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[1959] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[1960] Event Progression

[1961] When a user takes a specific action (e.g., heading to the castle), the device sends that action information to the server, which then checks the conditions for the event to occur. If the conditions are met, the server generates event information and sends it to the device. The device then displays the event information to the user, prompting them to take the next action.

[1962] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[1963] Achieving goals and distributing rewards

[1964] When a user achieves a game objective, the device sends that information to the server. The server verifies the achievement of the objective, generates reward information, and sends it to the device. The device displays the reward information to the user, and the game session ends.

[1965] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information (e.g., in-game currency), and sends it to the device. The device displays this to "User A" and notifies him that the game is over.

[1966] The above is a specific description of the embodiment for carrying out the present invention.

[1967] The processing flow will be explained below.

[1968] Program processing steps

[1969] Step 1:

[1970] User Login

[1971] The user opens the login form using the terminal.

[1972] The terminal presents the user with a form for entering a username and password.

[1973] The user enters their authentication information and clicks the Login button.

[1974] The device sends the authentication information to the server.

[1975] The server checks the authentication information against a database and determines whether the authentication is successful.

[1976] If the server is successful in the authentication, it generates a session ID and sends it to the terminal.

[1977] The terminal displays a successful login message.

[1978] Step 2:

[1979] Loading the Metaverse

[1980] The terminal sends a request for scene data and character data to the server.

[1981] The server retrieves scene data from the database based on the user's session ID and sends it to the terminal.

[1982] The server acquires the character data and sends it to the device.

[1983] The device renders the metaverse's 3D environment based on the scene data and character data it receives and displays it to the user.

[1984] Step 3:

[1985] Presentation of the game objective

[1986] The terminal requests game objective information from the server.

[1987] The server obtains the game objective information based on the user's session ID and sends it to the terminal.

[1988] The terminal displays the acquired game objective to the user.

[1989] Step 4:

[1990] Cooperation and information gathering

[1991] Users approach and interact with characters in the Metaverse.

[1992] The user inputs a question and sends it to the character.

[1993] The terminal transmits the query information to the server.

[1994] The server uses the character's AI engine to generate a response and send it to the device.

[1995] The terminal displays the response to the user.

[1996] Step 5:

[1997] Event Progression

[1998] Every time a user takes a specific action in the game, the device transmits the action information to the server.

[1999] The server receives the user's behavior information and checks the event occurrence conditions.

[2000] If the server satisfies the conditions, it generates event information and transmits it to the terminal.

[2001] The device displays event information to the user and prompts them to take the next action.

[2002] Step 6:

[2003] Achieving goals and distributing rewards

[2004] If it is determined that the user has achieved the objective of the game, the terminal transmits this information to the server.

[2005] The server verifies the achievement of the objectives and generates reward information.

[2006] The server transmits the reward information to the terminal.

[2007] The terminal displays reward information to the user and notifies them that the game session is ending.

[2008] Specific example explanation

[2009] As a concrete example, we will explain the specific actions that "User A" takes from logging in to achieving his or her goal.

[2010] Step 1:

[2011] "User A" uses a terminal to open the login form page, enters the username and password, and clicks the login button. The terminal sends the authentication information to the server, which checks the authentication information in the database. If the authentication is successful, the server generates a session ID and sends it back to the terminal. "User A"'s terminal displays a login success message.

[2012] Step 2:

[2013] After "User A" successfully logs in, the device sends a request for scene data and character data to the server. The server retrieves this data from the database and sends it to "User A's" device. The device renders the metaverse based on the received data and displays it to "User A."

[2014] Step 3:

[2015] "User A" operates the device to confirm the objective of the game, and the device requests the objective information from the server. The server obtains the objective information and sends it to the device. The device displays the information to "User A" and presents the objective of "find the lost treasure."

[2016] Step 4:

[2017] "User A" approaches a villager character (CP) and asks "Where is the treasure?" The terminal of "User A" sends the question information to the server, and the server generates an appropriate response (e.g., "It might be in the castle's basement") and sends it to the terminal. The terminal displays the response to "User A."

[2018] Step 5:

[2019] When "User A" heads towards the castle, the device sends this action information to the server. The server checks the conditions for the event to occur, and if the conditions are met, it generates event information that "the castle gates have been magically closed" and sends it to the device. The device then displays this event information to "User A" and prompts him to take the next action.

[2020] Step 6:

[2021] "User A" collects the necessary items and finds the treasure. The device sends the information to the server, which verifies that the objective was achieved. The server generates reward information (e.g., in-game currency) and sends it to the device. The device displays the reward information to "User A" and notifies him or her that the game session is ending.

[2022] Example 1

[2023] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2024] Current virtual reality systems require complex system processing after a user logs in, which can degrade the quality of the user experience. Furthermore, when users interact with characters in the virtual space, conventional systems only provide limited responses, resulting in a lack of completeness as a comprehensive gaming environment. Furthermore, there are many instances where manual processing is required for reward distribution when a user achieves a goal, and automation is needed.

[2025] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2026] In this invention, the server includes means for receiving and verifying authentication information for a user to log in to a virtual space, means for generating a session ID for the authenticated user, and means for generating question and answer prompts for a character in the virtual space using a generative AI model. This simplifies system processing after a user logs in, improving the quality of the user experience. It also enables more diverse and realistic responses in character interactions in the virtual space, improving the overall game environment. Furthermore, it automates reward allocation upon goal achievement, increasing overall efficiency.

[2027] "Authentication information" refers to a combination of a username and password that a user uses to log in to a virtual space.

[2028] A "server" is a central computer system for the entire virtual space system, providing data storage, processing, user authentication, and other functions.

[2029] A "session ID" is a unique identifier generated by the server to identify a user after the user has been authenticated.

[2030] A "terminal" is a device used by a user to access a virtual space, and includes a PC, smartphone, tablet, etc.

[2031] "Scene data" is data that constitutes the environment, background, objects, etc. within a virtual space.

[2032] "Character data" refers to data that includes information, attributes, actions, etc. of characters in a virtual space.

[2033] "Rendering" is the process of visually displaying a virtual 3D environment based on scene data and character data.

[2034] The "objective of the game" is the goal or task that the user must achieve within the virtual space.

[2035] A "generative AI model" is an algorithm or model that uses artificial intelligence technology to generate appropriate responses or content based on user input.

[2036] "Behavioral information" is data on the actions and choices made by the user within the virtual space.

[2037] An "event occurrence condition" is a condition or trigger element necessary for a specific event to occur in virtual space.

[2038] "Event information" is information that is generated when an event occurrence condition is met, and includes details of the event and the next action to be taken.

[2039] "Reward information" is details of the rewards a user will receive when they achieve a goal in the game. Examples include virtual currency, items, etc.

[2040] The present invention is a system in which a user logs into a virtual space and cooperates with friends and characters to gather information and progress through events in order to achieve a specific goal. Each component and its operation will be described in detail below.

[2041] User terminal

[2042] User devices include PCs, smartphones, tablets, and other devices. Users use these devices to access the virtual space and perform everything from login to gameplay. The devices use software such as HTML, JavaScript, React, and Three.js to execute the user interface and 3D rendering.

[2043] Server System

[2044] The server system is a central computer system that handles user authentication, database management, response generation using generative AI models, event management, etc. The server runs on software such as Node.js and MySQL.

[2045] Metaverse Database

[2046] The metaverse database stores user information, scene data, character data, game objectives, event information, etc. MySQL is used as the database management system.

[2047] Login process

[2048] The user displays a login form on their device and enters their username and password. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a new session ID and returns it to the device. The device notifies the user of successful authentication and displays an entry point to the virtual space.

[2049] As a concrete example, "User A" attempts to log in on a terminal. The terminal sends the input information of "User A" to the server, and after the server verifies the authentication information, it generates a session ID and sends it to "User A's" terminal.

[2050] Loading the Metaverse

[2051] After a user successfully logs in, the device sends a request for scene and character data to the server. The server retrieves this data from the database based on the user's session ID and sends it to the device. The device then renders a 3D virtual environment based on the received data and displays it to the user.

[2052] For example, after "User A" logs in, the device requests scene data and character information from the server. The server obtains this data and sends it to "User A's" device, which then displays the virtual space based on that data.

[2053] Presentation of the game objective

[2054] The device requests game objective information from the server. The server retrieves the objective information from a database based on the user's session ID. The server sends the objective information to the device, which displays it to the user.

[2055] As a concrete example, "User A" sends a request to the server to confirm the objective of the game. The server obtains the objective information, "find treasure," and sends it to "User A's" device. The device displays it, allowing "User A" to recognize the objective.

[2056] Cooperation and information gathering

[2057] Users can interact with other characters in the virtual space. When a user approaches a specific character (e.g., a villager) and asks a question, the device sends the question to the server. The server uses the character's AI engine (generative AI model) to generate an appropriate response and sends it to the device. The device then displays the response to the user.

[2058] For example, "User A" asks a villager character "Where is the treasure?" The device sends the question to the server, and the server generates a response that "It might be in the castle's basement" and sends it to the device. The device displays it to "User A."

[2059] Prompt Sentence Examples

[2060] User prompt: "Where is the treasure?"

[2061] Event Progression

[2062] When a user takes a specific action (e.g., heads to the castle), the device sends the action information to the server. The server checks the event occurrence conditions and generates event information if the conditions are met. The server then sends the event information to the device, which then displays it to the user.

[2063] For example, when "User A" heads to the castle, the device sends this action to the server. The server confirms that the conditions are met, generates event information indicating that the castle gates have been magically closed, and sends it to the device. The device then displays this information to "User A."

[2064] Achieving goals and distributing rewards

[2065] When the user achieves the game objective, the device sends that information to the server. The server confirms the achievement and generates reward information (e.g., in-game currency). The server then sends the reward information to the device, which displays it to the user. This ends the game session.

[2066] For example, if "User A" finds a treasure, the device sends that information to the server. The server confirms that the objective has been achieved, generates reward information, and sends it to the device. The device displays it to "User A" and notifies him that the game is over.

[2067] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2068] Step 1:

[2069] The user enters their username and password into the login form on their device.

[2070] Input: Username, Password

[2071] Specific behavior: The device displays the login screen using HTML and JavaScript. The user fills out the form and clicks the login button.

[2072] Step 2:

[2073] The terminal transmits the entered authentication information to the server.

[2074] Input: Username, Password

[2075] Specific operation: Sends authentication information to the server via an HTTP POST request using the JavaScript Fetch API.

[2076] Step 3:

[2077] The server checks the credentials against a database.

[2078] Input: Username, Password

[2079] Output: Authentication success or failure, session ID

[2080] What happens: The server uses Node.js to query the MySQL database for authentication information. If the credentials match, it generates a new session ID with a UUID.

[2081] Step 4:

[2082] The server returns the authentication result to the terminal and includes the new session ID.

[2083] Input: Authentication success or failure, session ID

[2084] Output: Session ID

[2085] Specific operation: If authentication is successful, the session ID is returned to the terminal in the HTTP response.

[2086] Step 5:

[2087] The terminal notifies the user of successful authentication and displays an entry point to the virtual space.

[2088] Input: Session ID

[2089] Specific behavior: The device displays a successful authentication message and an entry point link. The screen is updated using React.

[2090] Step 6:

[2091] The user clicks on an entry point into the virtual space.

[2092] Input: Entry point URL, Session ID

[2093] Specific operation: The terminal sends a request to the entry point to the server.

[2094] Step 7:

[2095] The terminal sends a request for scene data and character data to the server.

[2096] Input: Session ID

[2097] Specific operation: The device requests scene data and character data from the server using an HTTP GET request.

[2098] Step 8:

[2099] The server retrieves scene data and character data from the database and transmits them to the terminal.

[2100] Input: Session ID

[2101] Output: Scene data, character data

[2102] Specific operation: The server uses Node.js to retrieve the necessary data from the MySQL database and send it to the terminal as an HTTP response.

[2103] Step 9:

[2104] The device renders a 3D virtual environment based on the scene data and character data it receives and displays it to the user.

[2105] Input: Scene data, character data

[2106] What it does: The device uses Three.js to render the scene and characters in 3D and display them in the user interface.

[2107] Step 10:

[2108] The terminal requests information for game purposes from the server.

[2109] Input: Session ID

[2110] Specific operation: The device requests the game objective from the server via an HTTP GET request.

[2111] Step 11:

[2112] The server retrieves game objective information from the database based on the user's session ID and sends it to the terminal.

[2113] Input: Session ID

[2114] Output: Game objective information

[2115] Specific operation: The server uses Node.js to retrieve game objective information from the MySQL database and sends it to the terminal as an HTTP response.

[2116] Step 12:

[2117] The terminal displays the received game objective information to the user.

[2118] Input: Game objective information

[2119] Specific operation: The device uses React to display game objective information on the user interface.

[2120] Step 13:

[2121] The user sends a question to a character in the virtual space.

[2122] Input: User's question

[2123] Specific operation: The device sends the question to the server.

[2124] Step 14:

[2125] The terminal sends the user's question to the server.

[2126] Input: User's question

[2127] Specific operation: The question is sent to the server via an HTTP POST request, possibly using WebSocket.

[2128] Step 15:

[2129] The server uses an AI engine (generative AI model) to generate a response to the question and send it to the terminal.

[2130] Input: User's question

[2131] Output: Response by AI engine

[2132] Specific operation: The server uses a generative AI model (e.g., GPT-3) to generate an answer to the question and sends it to the device in an HTTP response.

[2133] Step 16:

[2134] The terminal displays the received response to the user.

[2135] Input: Response by AI engine

[2136] Specific behavior: The device displays the response in the user interface using React.

[2137] Step 17:

[2138] The user takes a specific action (e.g., heads to the castle).

[2139] Input: User behavior information

[2140] Specific operation: The device sends its behavior information to the server.

[2141] Step 18:

[2142] The terminal transmits the user's behavior information to the server.

[2143] Input: User behavior information

[2144] Specific operation: Sends behavioral information to the server via an HTTP POST request.

[2145] Step 19:

[2146] The server checks the event occurrence conditions, and if the conditions are met, generates event information and sends it to the terminal.

[2147] Input: User behavior information

[2148] Output: Event information

[2149] Specific operation: The server checks the event occurrence conditions based on the behavioral information, and if the conditions are met, it generates event information using Node.js and sends it to the terminal as an HTTP response.

[2150] Step 20:

[2151] The terminal displays the received event information to the user.

[2152] Input: Event information

[2153] Specific behavior: The device displays the event information on the user interface using React.

[2154] Step 21:

[2155] The user completes the game objective (e.g., finds treasure).

[2156] Input: Objective achievement information

[2157] Specific operation: The device sends the information to the server.

[2158] Step 22:

[2159] The terminal transmits the goal achievement information to the server.

[2160] Input: Objective achievement information

[2161] Specific behavior: Sends achievement information to the server using an HTTP POST request.

[2162] Step 23:

[2163] The server verifies the achievement of the objectives and generates reward information.

[2164] Input: Objective achievement information

[2165] Output: Reward information

[2166] Specific operation: The server confirms that the goal has been achieved and generates reward information using Node.js.

[2167] Step 24:

[2168] The server transmits the reward information to the terminal.

[2169] Input: Reward information

[2170] Specific operation: Reward information is sent to the terminal in an HTTP response.

[2171] Step 25:

[2172] The terminal displays the received reward information to the user.

[2173] Input: Reward information

[2174] Specific behavior: The device displays the reward information on the user inter...

Claims

1. a means for transmitting credentials for a user to log into the metaverse; means for the server to receive the authentication information and authenticate the user; a means for the server to generate a session ID for the authenticated user; A means for the terminal to request scene data and character data from the server; A means for the server to acquire scene data and character data and transmit them to the terminal; a means for rendering a metaverse based on the scene data and character data received by the device; a means for the terminal to request a game objective from the server and display the received objective; a means for a user to submit a question to a character in the Metaverse; A means for the server to generate a response to the query and transmit it to the terminal; means for displaying the response received by the terminal to the user; A means for transmitting user behavior information to a server and checking an event occurrence condition; means for generating and transmitting event information to the terminal when the server satisfies the conditions; means for displaying the event information received by the terminal to a user; A means for the server to verify the user's achievement of the goal and generate reward information; The system includes means for transmitting and displaying the reward information to a terminal.

2. The system of claim 1 , wherein the terminal comprises means for rendering the metaverse based on the scene data and the character data.

3. The system of claim 1, further comprising: means for a user to send a question to a character in the metaverse; and means for the server to generate a response to the question and send it to the terminal.

Images