System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a request for a specific era or event from a user; means for generating a virtual reality space that reproduces a past era using generative artificial intelligence based on the request; means for simulating behavior and conversation of a historical person in the virtual reality space; means for providing the virtual reality space to the user and transmitting the virtual reality space to a device with which the user interacts; and means for generating a response to the user's interaction in real time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With current technology, it is impossible to travel back in time and directly interact with historical events and people. However, many people wish to interact with historical events and people. Furthermore, constructing a virtual reality space requires a significant amount of man-hours, making it difficult to efficiently reproduce such experiences with current technology. Furthermore, simulating real-time conversations and actions within a virtual reality space is technically difficult, making it challenging to provide a realistic experience. [Means for solving the problem]

[0005] The present invention provides a system including: means for receiving a request for a specific era or event from a user; means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request; means for simulating the behavior and conversation of historical figures within the virtual reality space; means for providing the virtual reality space to a user and transmitting the virtual reality space to a device with which the user interacts; and means for generating responses to the user's interactions in real time, thereby enabling the user to realistically experience a past era and enjoy direct communication with historical figures.

[0006] A "user" is an individual or group that uses the system to experience past eras and events and interact with historical figures in a virtual reality space.

[0007] A "request" is input information that allows a user to specify a particular era or event.

[0008] "Generative artificial intelligence" is an AI technology that generates necessary data and content based on input information.

[0009] A "virtual reality space" is a computer-generated, three-dimensional simulation environment that users can experience as if it were real.

[0010] "System" refers to a set of devices or software that receives user requests, generates and provides a virtual reality space, and manages the simulation of historical figures and user interaction.

[0011] "Historical figures" are characters in a virtual reality space generated based on a specified era or event.

[0012] "Behavior" refers to the actions and movements of historical figures within the virtual reality space.

[0013] A "conversation" is a verbal exchange between a historical figure and a user in a virtual reality space.

[0014] "Simulation" is a technological operation that realistically reproduces the actions and conversations of historical figures within a virtual reality space.

[0015] "Real-time" refers to a processing process that reacts immediately to user input and generates and provides a response without delay.

[0016] A "device" is the hardware (e.g., VR headset and controllers) that a user uses to experience a virtual reality space.

[0017] "Analysis" is the process of understanding a user's request or input information and generating appropriate data or responses based on that. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures. The system of the present invention is implemented in the following manner.

[0040] System Overview

[0041] The system receives requests from users for specific eras or events, uses generative artificial intelligence to generate a virtual reality space based on the request, simulates historical figures within the virtual reality space, and provides the virtual reality space to a device for user interaction. It then generates real-time responses to the user's actual actions within the virtual reality space.

[0042] Program processing

[0043] 1. Receiving a user request

[0044] The terminal receives a request from the user selecting a past era or event and transmits it to the server.

[0045] Example: A user requests Japan during the Sengoku period.

[0046] 2. Creation of virtual reality space

[0047] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[0048] Example: The server generates backgrounds, scenery, and buildings for Japan during the Sengoku period.

[0049] 3. Simulation of historical figures

[0050] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[0051] Example: The server specifies the samurai's behavior patterns and conversation content.

[0052] 4. Provision of virtual reality space

[0053] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[0054] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[0055] 5. Generating real-time responses

[0056] The server generates responses from historical figures in real time based on the user's actions.

[0057] Example: When a user speaks to a samurai, the server uses generative artificial intelligence to generate the samurai's response and sends it to the device.

[0058] Specific examples

[0059] Example 1: Interacting with a samurai in Sengoku period Japan

[0060] 1. User Request

[0061] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[0062] 2. Creation of virtual reality space

[0063] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[0064] 3. Simulation of historical figures

[0065] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[0066] 4. Provision of virtual reality space

[0067] The device provides the user with a realistic virtual reality space via a VR headset.

[0068] 5. Interaction and real-time response

[0069] When the user talks to the samurai, the server generates the samurai's response in real time and sends it to the terminal. The samurai will respond with something like, "Can you tell me about your experience in today's battle?"

[0070] In this way, users can realistically experience past eras and enjoy conversations with historical figures. The present invention provides a system that allows users to easily experience visiting past eras.

[0071] The processing flow will be explained below.

[0072] Step 1:

[0073] The user launches an application.

[0074] Action: The device confirms that the application is running and displays the login screen.

[0075] Step 2:

[0076] The user enters their login information and logs in.

[0077] Operation: The device sends the user's login information to the server and performs the authentication process.

[0078] Step 3:

[0079] The user selects a particular era or event.

[0080] How it works: The device receives the user's selection through the UI and sends a request to the server.

[0081] Step 4:

[0082] The server receives and parses the request.

[0083] How it works: The server analyzes the user's request and identifies the required data.

[0084] Step 5:

[0085] The server activates generative artificial intelligence and generates a virtual reality space.

[0086] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[0087] Step 6:

[0088] The server formats the virtual reality space and converts it into a VR-compatible data format.

[0089] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[0090] Step 7:

[0091] The server generates data on historical figures.

[0092] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[0093] Step 8:

[0094] The server generates character dialogue scripts using generative artificial intelligence.

[0095] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[0096] Step 9:

[0097] The server sends the VR data to the device.

[0098] Operation: The server sends the generated VR space data to the device.

[0099] Step 10:

[0100] The device transfers the data to the VR headset.

[0101] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[0102] Step 11:

[0103] The user interacts with the VR headset and controllers.

[0104] Actions: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with characters.

[0105] Step 12:

[0106] The device detects the user's actions and sends them to the server.

[0107] How it works: The device detects the user's actions (e.g. talking to the samurai) and sends that information to the server.

[0108] Step 13:

[0109] The server analyzes the user's behavior and generates an appropriate response.

[0110] How it works: The server analyzes the user's behavior and uses generative artificial intelligence to generate real-time responses from historical figures.

[0111] Step 14:

[0112] The server sends the generated response data to the terminal.

[0113] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[0114] Step 15:

[0115] The terminal displays the response data to the user.

[0116] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[0117] Example 1

[0118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] Conventional virtual reality systems, when users experience past eras or historical events, have problems with insufficient real-time interaction and response, and limited dialogue and behavior with historical characters. Furthermore, there is a lack of technology to generate highly accurate virtual reality spaces based on user requests. To address these issues, a system that can achieve real-time response and precise simulation by utilizing generative artificial intelligence is needed.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, means for simulating the behavior and conversation of historical figures in the virtual reality space, means for providing the virtual reality space to the user and transmitting the virtual reality space to a device with which the user interacts, means for monitoring interaction data from the user and generating responses in real time, means for generating 3D models of historical backgrounds, scenery, and buildings using generative artificial intelligence, and means for transmitting data of the generated virtual reality space to the user's device, thereby enabling the user to realistically experience past eras and enjoy conversations with historical figures.

[0122] "User" refers to any individual or entity that uses the System.

[0123] A "request" is a request in the form of a user inputting into a terminal their intention to experience a specific era or event.

[0124] "Terminal" refers to the device through which a user accesses the system, inputs requests, and experiences the virtual reality space, including, but not limited to, a smartphone, PC, or tablet.

[0125] "Server" refers to a central computer or cloud service that receives requests from users, uses generative artificial intelligence to simulate virtual reality spaces and historical figures, and sends the results to terminals.

[0126] "Generative AI" refers to AI technology that generates the actions and conversations of past eras and historical figures based on user requests. Specifically, it includes machine learning models and natural language processing models.

[0127] "Virtual reality space" refers to a 3D computer graphics environment created using generative artificial intelligence that recreates a specific era or event.

[0128] "Historical figures" refer to characters who actually existed in the past or who are spoken of as legends, and within this system their actions and conversations are simulated by generative artificial intelligence.

[0129] "Behavior" refers to the actions and movements of historical figures within the virtual reality space.

[0130] "Conversation" refers to a dialogue between a historical figure and the user or another character within a virtual reality space.

[0131] "Simulation" refers to the process of realistically recreating the behavior and conversations of historical figures in a virtual reality space using generative artificial intelligence.

[0132] "Interaction" refers to the operations and actions performed by the user within the virtual reality space, including the system's response to them.

[0133] "Real-time responses" refer to the responses and actions of historical figures that are generated instantly in response to user interactions.

[0134] "3D model" refers to a three-dimensional computer graphics object created by generative artificial intelligence, and specifically includes backgrounds, landscapes, buildings, etc.

[0135] The system of the present invention allows users to realistically experience past eras and enjoy direct communication with historical figures. Specific embodiments for carrying out the present invention will now be described in detail.

[0136] First, a user inputs a request to experience a specific era or event into a terminal. The terminal can be a smartphone, PC, tablet, or other device, and sends the user's input as an HTTP request to a server. The server then receives this request and generates a virtual reality space using generative artificial intelligence. Generative artificial intelligence uses machine learning models and natural language processing models, such as OpenAI's GPT model.

[0137] Based on user requests, the server retrieves information on historical background, landscapes, buildings, and other aspects, and performs advanced 3D modeling. Game engines such as Unity and Unreal Engine are used for this. The server also uses generative artificial intelligence to simulate the behavior and conversation of historical figures in the virtual reality space. The simulation includes characters such as samurai, feudal lords, and farmers.

[0138] The generated virtual reality space data is sent from the server to the user's device. The user wears a VR headset (e.g., Oculus Rift or HTC Vive) and experiences the virtual reality space. The data displayed on the VR headset includes Japanese landscapes, castles, and feudal lord mansions from the Warring States period, and the user can interact with them using the VR controller.

[0139] When a user performs a specific action in the virtual reality space, that data is sent to the server in real time, and the server uses generative artificial intelligence to generate an appropriate response. For example, when a user speaks to a samurai, the server generates the samurai's response and sends it to the device. If the user asks the samurai, "Will you tell me about your experience in today's battle?", the samurai can instantly generate and display a response such as, "I'll tell you about my experience in today's battle."

[0140] Specific examples

[0141] Example 1: Interacting with a samurai in Sengoku period Japan

[0142] The user inputs a request into the device saying, "I want to interact with a samurai in Sengoku-period Japan." This request is sent to the server as an HTTP request. The server uses generative artificial intelligence to generate Sengoku-period scenery, castles, and feudal lord mansions. The generated virtual reality space data is sent to the device, and the user experiences the virtual reality space using a VR headset. When the user asks the samurai, "Will you tell me about your experience in today's battle?", the server uses generative artificial intelligence to generate a response from the samurai, and the response "I'll tell you about my experience in today's battle" is displayed.

[0143] Prompt Sentence Examples

[0144] "I want to talk to a samurai from the Sengoku period. I want to learn his techniques."

[0145] "I would like to see a feudal lord's mansion. I would like to know what kind of decorations and furniture were there."

[0146] "I want to experience a day in the life of a farmer. I want to see what kind of work they do."

[0147] In this way, users can experience a past era in a realistic way and enjoy conversations with historical figures. This invention allows users to easily experience visiting a past era.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Step 1:

[0150] Receiving a user request

[0151] Input: The user types a request into the terminal to select a particular era or event.

[0152] Specific operation: The user inputs a request saying, "I want to interact with a samurai in Japan during the Warring States period."

[0153] Output: The terminal sends the user's request to the server as an HTTP request.

[0154] Step 2:

[0155] Processing virtual reality space generation requests

[0156] Input: The user request sent from the terminal.

[0157] Specific operation: The server receives the request and constructs a prompt to generate the virtual reality space using generative artificial intelligence (e.g., OpenAI's GPT model).

[0158] Output: Constructing prompts for generative artificial intelligence.

[0159] Step 3:

[0160] Virtual reality space generation

[0161] Input: A prompt for generative artificial intelligence.

[0162] How it works: The server generates 3D models of Japanese landscapes, castles, feudal lord mansions, etc. from the Warring States period based on prompts, using game engines such as Unity or Unreal Engine.

[0163] Output: 3D model data of the generated virtual reality space.

[0164] Step 4:

[0165] Simulation of historical figures

[0166] Input: 3D model data of the virtual reality space and additional prompt data.

[0167] Specific operation: The server uses generative artificial intelligence to generate the behavior and conversation scripts of historical figures in the virtual reality space. It generates the behavior patterns and conversation content of characters such as samurai, feudal lords, and farmers.

[0168] Output: Simulation data of the generated historical figures.

[0169] Step 5:

[0170] Integration and provision of virtual reality spaces

[0171] Input: 3D model data of virtual reality space and simulation data of historical figures.

[0172] How it works: The server aggregates the virtual reality data and sends it to the user's device, which then provides the virtual reality space to the user via a VR headset.

[0173] Output: The virtual reality space is displayed on the user's device.

[0174] Step 6:

[0175] Monitor interactions and generate real-time responses

[0176] Input: User behavior data (e.g., voice, movement).

[0177] How it works: The device sends user behavior data to the server, which then uses generative artificial intelligence to generate real-time responses based on the user's behavior. For example, when the user speaks to the samurai, the voice data is sent to the server, which then generates an appropriate response from the samurai.

[0178] Output: The generated real-time response data is sent to the terminal and displayed as an appropriate character's response in the virtual reality space.

[0179] As a specific example, if a user requests, "I want to interact with a samurai in Japan during the Warring States period," the user will go through the above steps to be immersed in a virtual reality space of the Warring States period and be able to enjoy real-time conversations with historical figures. This system makes it easy to experience visiting a past era.

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] Conventional virtual reality experience systems lacked the technology to enable real-time interaction with past eras or historical figures. In particular, it was difficult to generate intuitive and natural responses to user actions and interactions, limiting their ability to provide a realistic experience. Furthermore, there was no easy way to achieve such an experience using mobile devices such as smartphones. This resulted in a lack of interactivity in historical learning and entertainment.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era based on the request using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This enables a means for experiencing the virtual reality space using a smartphone and providing an interface for interacting with characters, and means for analyzing the conversation between the historical figure and the user using a natural language processing model and generating an appropriate response based on the content of the conversation.

[0185] definition statement

[0186] The "means for receiving a request from a user for a specific era or event" refers to a means for receiving the request within the system through an interface for a user to input a request for a past era or a specific event.

[0187] "Generative AI" is AI that has the ability to generate new data based on large amounts of data according to specified parameters.

[0188] A "means for generating a virtual reality space" is a means for recreating a past era on a computer based on a received request, and constructing a virtual environment that can be used visually and experientially by the user.

[0189] "Means for simulating the behavior and conversation of historical figures" refers to a means of imitating the behavior patterns and speaking styles of people who lived in past eras using generative artificial intelligence, enabling interaction with users.

[0190] "Devices for user interaction" are devices or interfaces that allow users to access a virtual reality space and perform operations and interactions within that space.

[0191] "Means for generating responses in real time" refers to a means for instantly generating appropriate responses to the user's actions and statements, and replying as a historical figure within the virtual reality space.

[0192] "Means for providing an interface for experiencing a virtual reality space using a smartphone and interacting with characters" refers to a means for displaying a virtual reality space using a smartphone as a medium and for users to interact with characters within that space.

[0193] A "natural language processing model" is an artificial intelligence-based model for analyzing human language and understanding and generating its meaning and context.

[0194] "Means for generating appropriate responses" refers to techniques for providing natural and relevant replies based on the content of the dialogue with the user.

[0195] System configuration

[0196] This invention is composed of a user, a server, and a terminal. The user operates the system using a smartphone to request a specific era or event.

[0197] Program processing

[0198] 1. Receiving a user request

[0199] Users use a smartphone interface to request a specific time period or event from the past, which is then sent over the internet to a server.

[0200] 2. Creation of virtual reality space

[0201] The server uses OpenAI's generative AI model (e.g., GPT-3) to generate a virtual reality space based on the user's request, faithfully recreating historical and cultural elements.

[0202] 3. Simulation of historical figures

[0203] The server also uses generative AI models to simulate the behavior and conversation of historical figures, referencing a database of past events to ensure the simulated characters act and speak in a way appropriate to the time period.

[0204] 4. Provision of virtual reality space

[0205] The virtual reality space and simulated historical figures generated by the server are sent to the user's smartphone, where the user can access the virtual reality space and experience it using devices such as a VR headset and controller.

[0206] 5. Generating real-time responses

[0207] The server generates real-time responses to user interactions. When a user speaks to a historical figure, the content is analyzed using a natural language processing model, and an appropriate response is returned.

[0208] Hardware and Software

[0209] Hardware: Smartphone (iOS or Android compatible), VR headset

[0210] software:

[0211] OpenAI GPT-3 API: Used as a generative AI model to generate historical context and character behavior.

[0212] Unity3D: A platform and engine for building VR environments.

[0213] REST API: An interface for data communication between the server and the app.

[0214] Specific examples

[0215] If the user requests to interact with a warlord in Sengoku-era Japan, the following prompt sentence is used:

[0216] I would like to talk to warlords in Japan during the Sengoku period and learn about their daily lives and battles.

[0217] Based on this request, the system recreates the background, scenery, castles, and mansions of the Sengoku period, and generates the behavioral patterns and conversations of the warlords. When the user enters the virtual reality space using a VR headset and talks to the warlord, the warlord will respond in real time with a response such as, "Today was a tough battle. I fought and risked my life in it. It's time to show true courage."

[0218] This invention provides a system that allows users to easily experience visiting past eras in a realistic way using a smartphone and a generative AI model.

[0219] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0220] Program processing steps

[0221] Step 1:

[0222] Receiving a user request

[0223] explanation

[0224] Users launch the smartphone application and input the past era or event they wish to experience. This input is done by sending a request to the application screen, such as "I want to experience Japan during the Warring States period." The request is then sent to the server through the application interface.

[0225] Input and Output

[0226] Input: A request typed by a user through a smartphone interface (e.g., "I want to experience Japan during the Warring States period").

[0227] Output: The request data sent to the server.

[0228] operation

[0229] Specific operation: The user enters the desired era and event in the input form and presses the "Submit" button. The application sends the request data to the server.

[0230] Step 2:

[0231] Virtual reality space generation

[0232] explanation

[0233] The server processes the received request and sends a prompt to OpenAI's generative AI model (GPT-3). The prompt conveys details of the requested era and event to the generative AI model. The generative AI model generates background data for the virtual reality space based on the prompt.

[0234] Input and Output

[0235] Input: The request data received by the server and a prompt (e.g., "Recreate the setting and key events of Japan during the Warring States period.").

[0236] Output: Background data of the generated virtual reality space.

[0237] operation

[0238] How it works: The server takes in the request data, generates a prompt based on it, and sends it to the OpenAI API. The generative AI model generates background data for the virtual reality space in response to the prompt and returns it to the server.

[0239] Step 3:

[0240] Simulation of historical figures

[0241] explanation

[0242] The server uses a generative AI model to generate behavior and conversation data for historical figures, which involves referencing a historical database to set behavior patterns and conversation content appropriate for that era.

[0243] Input and Output

[0244] Input: Background data of the virtual reality space generated by the server and a prompt (e.g., "Generate the behavior patterns and speech patterns of a warlord from the Sengoku period.").

[0245] Output: Behavioral and conversational data of the generated historical figures.

[0246] operation

[0247] Specific operation: The server generates a more detailed prompt sentence and sends it to the generative AI model. The generative AI model generates behavior and conversation data of the historical figure based on the prompt and replies to the server.

[0248] Step 4:

[0249] Providing virtual reality space

[0250] explanation

[0251] The server sends the generated virtual reality space and simulation data to the user's smartphone, where the user can access the virtual reality space using an application.

[0252] Input and Output

[0253] Input: Generated virtual reality space and simulation data of historical figures.

[0254] Output: VR data sent to the user's smartphone.

[0255] operation

[0256] Specific operation: The server packages the generated virtual reality space data and sends it to the user's smartphone using a REST API. The user then accesses the virtual reality space through the smartphone screen and begins the experience.

[0257] Step 5:

[0258] Generate real-time responses

[0259] explanation

[0260] When a user interacts with a historical figure in the virtual reality space, the content of the interaction is transmitted to the server, which then analyzes the interaction using a natural language processing model and generates an appropriate response. The generated response is then transmitted in real time to the user's smartphone and displayed in the virtual reality space.

[0261] Input and Output

[0262] Input: User interaction (e.g., "How was your battle today?").

[0263] Output: The generated response of the historical figure (e.g., "Today was a tough battle. I risked my life in it. It's time to show true courage.").

[0264] operation

[0265] How it works: When a user speaks through the microphone in their VR headset, the voice data is sent to the server. The server uses a natural language processing model to convert the voice data into text, analyzes the text, and sends it to the generative AI model. The generative AI model generates a response text and sends it back to the server. The server then sends this response to the user's smartphone and displays it in the virtual reality space.

[0266] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0267] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures, and further relates to a system that provides a more natural and realistic experience by combining it with an emotion engine that recognizes the user's emotions. The system of the present invention is implemented in the following form.

[0268] System Overview

[0269] This system receives requests from users for specific eras or events, and uses generative artificial intelligence to generate a virtual reality space based on the request. It then simulates historical figures within the virtual reality space and provides the virtual reality space to a device for user interaction. It also uses an emotion engine to recognize the user's emotions, and adjusts the responses and behavior of the historical figures in the virtual reality space in real time based on the recognition results, providing a more emotionally relevant experience for the user.

[0270] Program processing

[0271] 1. Receiving a user request

[0272] The user selects a past era or event and sends a request from the terminal to the server.

[0273] Example: A user requests Japan during the Sengoku period.

[0274] 2. Creation of virtual reality space

[0275] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[0276] Example: The server generates the scenery, buildings and backgrounds of Japan during the Sengoku period.

[0277] 3. Simulation of historical figures

[0278] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[0279] Example: The server sets the samurai's behavior patterns and conversation content.

[0280] 4. Provision of virtual reality space

[0281] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[0282] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[0283] 5. Emotion Recognition and Analysis

[0284] The device uses an emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data.

[0285] Example: The device recognizes emotions such as joy, surprise, and sadness from the user's tone of voice and facial expressions.

[0286] 6. Generating real-time responses

[0287] The server generates real-time responses of historical figures based on the user's behavioral and emotional data.

[0288] Example: The user asks the samurai a question, and the samurai responds kindly according to the user's emotions.

[0289] Specific examples

[0290] Example 1: Interacting with a samurai in Sengoku period Japan

[0291] 1. User Request

[0292] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[0293] 2. Creation of virtual reality space

[0294] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[0295] 3. Simulation of historical figures

[0296] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[0297] 4. Provision of virtual reality space

[0298] The device provides the user with a realistic virtual reality space via a VR headset.

[0299] 5. Emotion Recognition and Analysis

[0300] The user asks the samurai a question, and the device uses an emotion engine to analyze the tone of voice and facial expressions to obtain emotional data.

[0301] 6. Interaction and real-time response

[0302] The server generates the samurai's response in real time based on the user's emotional data and provides it to the user via the terminal.

[0303] Example: If a user asks the samurai with a smile, "How can I become stronger?", the samurai will reply, "Your enthusiasm is already proof of your strength."

[0304] In this way, users can not only experience a past era realistically and converse with historical figures, but also enjoy a more emotionally responsive and realistic experience by adjusting their responses and behavior according to the user's emotions. The present invention realizes a system that allows users to easily experience visiting a past era while providing an emotionally rich experience.

[0305] The processing flow will be explained below.

[0306] Step 1:

[0307] The user launches an application.

[0308] Action: The device confirms that the application is running and displays the login screen.

[0309] Step 2:

[0310] The user enters their login information and logs in.

[0311] Operation: The device sends the user's login information to the server and performs the authentication process.

[0312] Step 3:

[0313] The user selects a particular era or event.

[0314] How it works: The device receives the user's selection through the UI and sends a request to the server.

[0315] Step 4:

[0316] The server receives and parses the request.

[0317] How it works: The server analyzes the user's request and identifies the required data.

[0318] Step 5:

[0319] The server activates generative artificial intelligence and generates a virtual reality space.

[0320] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[0321] Step 6:

[0322] The server formats the virtual reality space and converts it into a VR-compatible data format.

[0323] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[0324] Step 7:

[0325] The server generates data on historical figures.

[0326] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[0327] Step 8:

[0328] The server generates character dialogue scripts using generative artificial intelligence.

[0329] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[0330] Step 9:

[0331] The server sends the VR data to the device.

[0332] Operation: The server sends the generated VR space data to the device.

[0333] Step 10:

[0334] The device transfers the data to the VR headset.

[0335] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[0336] Step 11:

[0337] The user interacts with the VR headset and controllers.

[0338] How it works: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with the characters.

[0339] Step 12:

[0340] The device analyzes the user's voice and facial expressions using an emotion engine.

[0341] How it works: The device analyzes emotional data from the user's tone of voice and facial expressions and sends that data to the server.

[0342] Step 13:

[0343] The server analyzes the user's behavioral and emotional data.

[0344] How it works: The server comprehensively analyzes the user's behavior and emotional data to determine the appropriate response of the historical figure.

[0345] Step 14:

[0346] The server sends the generated response data to the terminal.

[0347] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[0348] Step 15:

[0349] The terminal displays the response data to the user.

[0350] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[0351] Example 2

[0352] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0353] In conventional virtual reality systems, when users experience past eras, dialogue and interactions with historical figures cannot be adjusted in real time, resulting in a lack of realism and immersion. Furthermore, there was no system that could generate responses based on the user's emotional state, making it difficult to provide an experience that is more in tune with each individual user's emotions. Furthermore, the technical challenge of analyzing emotional data such as the user's voice and facial expressions and generating corresponding responses in real time remained unresolved.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This allows the user to realistically experience a past era, adjust dialogue and interaction with historical figures in real time, and generate responses according to the user's emotional state.

[0355] The "user request receiving means" is a means for the user to select a specific era or event and transmit that information to the system.

[0356] "Generative AI" is an AI technology used to recreate past eras, and is an AI capable of generating virtual reality spaces based on requests.

[0357] A "virtual reality space generation means" is a means for constructing a virtual reality space that recreates a specific era or event using generative artificial intelligence.

[0358] A "means for simulating historical figures" is a means for simulating the behavior and conversation of historical figures in a virtual reality space, providing users with realistic interactions.

[0359] The "virtual reality space providing means" refers to a device or method that provides the generated virtual reality space to the user and allows the user to interact within that space.

[0360] The "emotion engine" is an engine that analyzes the user's voice, facial expressions, body movements, etc., and recognizes their emotional state.

[0361] The "emotion data acquisition means" is a means for acquiring the user's emotion data analyzed using the emotion engine.

[0362] The "real-time response generation means" is a means for generating responses of characters in a virtual reality space in real time based on the user's behavior and emotional data.

[0363] The present invention relates to a system that allows users to experience past eras in a realistic way and enjoy direct communication with historical figures. Furthermore, the system provides a more natural and immersive experience by incorporating an emotion engine that recognizes the user's emotions. This system is implemented as follows.

[0364] System Overview

[0365] This system first receives a request for a specific era or event from the user. The user operates the terminal, selects a specific era or event from the past, and sends the request to the server. For example, a request might be, "I want to talk to a samurai in Japan during the Warring States period."

[0366] The server then uses generative artificial intelligence (e.g., GPT-4) to generate a virtual reality space based on the request. Specifically, it uses 3D rendering technology and generative AI models to create the scenery, buildings, and backgrounds of the requested era. For example, it generates a Japanese landscape, castle, or feudal lord's mansion from the Warring States period.

[0367] The server then generates behavior and conversation scripts for historical figures (such as samurai, feudal lords, and farmers) who will be active in the virtual reality space. This uses natural language processing technology to simulate realistic conversations. For example, it generates an object called a "samurai" and sets its behavioral patterns. The behavioral patterns include actions such as "patrolling the castle in the morning" and "talking when the user approaches."

[0368] The generated virtual reality space is provided to the user via a terminal. The user can move freely within the virtual space using a VR headset and controllers and interact with historical figures. For example, the user wears a VR headset and uses the controllers in their hands to walk and touch within the virtual space. The terminal renders the video and audio in real time.

[0369] The system also includes an emotion engine. The device uses the emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data. For example, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine then analyzes. The analyzed emotional data includes emotions such as joy, surprise, and sadness.

[0370] Finally, the server generates real-time responses from historical figures in the virtual reality space based on the user's behavioral and emotional data. For example, if the user shows a surprised expression, the samurai might respond, "Did I surprise you?" In this way, users can realistically experience past eras and feel a sense of presence when interacting with historical figures.

[0371] As a concrete example, consider the case where a user sends a request from a device saying, "I want to interact with a samurai in Sengoku-period Japan." In this case, the server generates the scenery, castles, and feudal lord mansions of Sengoku-period Japan, and sets up the actions and conversations of samurai, peasants, and feudal lord characters in the virtual reality space. The device then provides the user with a realistic virtual reality space via a VR headset, and when the user asks the samurai a question, the device obtains emotional data from the user's tone of voice and facial expressions, and generates the samurai's response in real time based on that data.

[0372] As a result, users can not only visit past eras, but also enjoy a more emotionally responsive and realistic experience, as responses and behaviors are adjusted according to the user's emotions.

[0373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0374] Step 1:

[0375] Receiving a user request

[0376] The user selects a specific time period or event in the past and sends a request from the device to the server. The input is the time period or event selected by the user (e.g., "Japan during the Warring States period"). This input data is sent to the server via the device. Specifically, when the user makes a selection using the touchscreen and presses the confirmation button, the device forwards this to the server as an HTTP request. The output is data that the server has received and analyzed the request content.

[0377] Step 2:

[0378] Virtual reality space generation

[0379] The server analyzes the request and generates a virtual reality space using generative artificial intelligence (e.g., GPT-4). The input is information about a specific era or event received from the user. Based on this information, the server generates data for 3D rendering of landscapes, buildings, backgrounds, etc. Specifically, after analyzing the request, the server retrieves a dataset corresponding to "Japan during the Warring States period" and generates virtual castles and landscapes using a generative AI model. The output is data for the generated virtual reality space.

[0380] Step 3:

[0381] Simulation of historical figures

[0382] The server generates the behavior and conversation scripts of historical figures active within the virtual reality space. The inputs include data on the generated virtual reality space and information about the historical figures. Based on this, the server uses natural language processing technology to simulate realistic conversations. In terms of specific operations, the server generates an object called a "samurai" and sets its behavior patterns (e.g., "patrol the castle in the morning" or "talk to the user when they approach"). The output is the behavior patterns and conversation scripts of the generated historical figures.

[0383] Step 4:

[0384] Providing virtual reality space

[0385] The device provides the generated virtual reality space to the user. The input is virtual reality space data sent from the server. The device transfers this data to a VR headset and controller, allowing the user to move freely within the virtual space and interact with historical figures. Specifically, the user wears a VR headset and uses the controller in their hand to walk and touch within the virtual space. The output is the virtual reality space that the user actually experiences.

[0386] Step 5:

[0387] Emotion Recognition and Analysis

[0388] The device analyzes the user's voice, facial expressions, and body movements using an emotion engine to obtain emotion data. The input data includes the user's voice, facial expressions, and body movements. Specifically, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine analyzes (e.g., if the user smiles, it is recognized as "joy"). The output is the analyzed emotion data.

[0389] Step 6:

[0390] Generate real-time responses

[0391] The server generates a real-time response from the historical figure based on the user's behavior and emotion data. The input is the user's behavior data and emotion data. Specifically, the server receives the data sent from the emotion engine and adjusts the content of the historical figure's remarks (for example, if the user shows a surprised expression, the samurai will respond, "Did I surprise you?"). The output is the historical figure's real-time response.

[0392] This allows users to not only experience past eras in a realistic way and interact with historical figures, but also enjoy a more emotionally immersive experience as their responses and behavior are adjusted according to the user's emotions.

[0393] (Application example 2)

[0394] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0395] Conventional virtual reality systems lack emotional responses and interactions when users experience past times, making the experience less immersive. Additionally, factories and other historical sites tend to rely solely on modern technology and data to provide explanations, lacking in consideration of historical context and user emotions.

[0396] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0397] In this invention, the server includes means for receiving requests for specific eras or events from users, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures within the virtual reality space, thereby enabling users to experience a past era with a sense of realism that includes emotions.

[0398] The "means for receiving a request for a specific era or event from a user" is an interface or device that allows a user to select and request a specific era or event that the user wishes to experience.

[0399] "Generative AI" is an AI technology that automatically generates past eras and scenarios based on user requests.

[0400] "Means for generating virtual reality spaces" refers to devices or software that use generative artificial intelligence to create three-dimensional virtual spaces that faithfully recreate past eras and events.

[0401] "Means for simulating the behavior and conversation of historical figures within a virtual reality space" refers to devices or software for simulating in real time the movements and conversation of historical figures appearing within a virtual reality space.

[0402] "Means for providing a virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with" means means for providing the generated virtual reality space to a user and transmitting data to a headset or other device for the user to experience and interact within the space.

[0403] An "emotion engine" is software that analyzes and recognizes emotions from a user's voice, facial expressions, actions, etc.

[0404] The "means for generating responses of historical figures in real time based on emotion data" is a technology for generating responses and behaviors of historical figures in real time based on the emotion data of a user.

[0405] This invention relates to a virtual reality system that allows users to experience past eras realistically and interact with historical figures. By combining it with an emotion engine, it provides real-time responses and behaviors according to the user's emotions, realizing a more natural and immersive experience.

[0406] System Program Overview

[0407] The server receives requests from users for specific eras or events. For example, if a user requests a "factory tour from the early 20th century," the server analyzes the request and uses generative artificial intelligence to generate a virtual reality space. This virtual reality space realistically reproduces the factory's scenery, the operation of machines, and the behavior of workers.

[0408] The server then simulates the behavior and conversation of historical figures (in this case, factory workers and engineers) within the virtual reality space, enabling realistic interactions within the virtual reality space experienced by the user.

[0409] The generated virtual reality space is provided to the user via a VR headset and controller, allowing the user to experience a past era.

[0410] Hardware and Software Details

[0411] VR headset: A device that provides users with a virtual reality space.

[0412] Robot guide: A robot for interacting with users.

[0413] Emotion recognition sensor: A sensor that acquires emotions from the user's voice, facial expressions, and movements.

[0414] OpenAI GPT-3: As a generative artificial intelligence, it is used to analyze user requests and generate real-time responses based on virtual reality spatial data and emotions.

[0415] Emotion Recognition Library: A library for analyzing user emotions and obtaining emotional data.

[0416] VR Renderer: Rendering software for generating virtual reality spaces.

[0417] Robot Controller software: Software for controlling the operation of the robot guide.

[0418] Specific examples

[0419] For example, if a user requests "I want to experience a factory tour from the early 20th century," the server generates a virtual reality space using the following prompts:

[0420] "A user requests a tour of an early 20th century factory. What kind of scene would you generate?"

[0421] In the generated virtual reality space, old machinery and work scenes in the factory are realistically reproduced. If the user is excited, the emotion recognition sensor will recognize that emotion and the emotion engine will analyze the data. For example, if the user excitedly asks, "How does this machine work?", the robot guide can respond, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[0422] In this way, a system is realized that allows users to experience a past era with a sense of realism while engaging in emotional interaction.

[0423] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0424] Step 1:

[0425] The user inputs a request to experience a specific era or event. Specifically, the user sends a request via their terminal saying, "I would like to experience a factory tour from the early 20th century." This input data is sent to the server, which receives and analyzes the request.

[0426] Step 2:

[0427] Based on the received request, the server generates prompts to generate a virtual reality space using a generative AI model (OpenAI GPT-3). The input is the user's request, and the output is virtual data that defines the specific scenario and components of the virtual reality space. For example, it generates a prompt such as, "The user has requested a factory tour from the early 20th century. What kind of scene would you like to generate?"

[0428] Step 3:

[0429] The server inputs prompts into the generative AI model to generate details of the virtual reality space. The input is the prompt sentence, and the output is detailed virtual reality space data (e.g., factory scenery, machine operation, worker behavior, etc.). The server passes this output to the VR Renderer, which renders the virtual reality space.

[0430] Step 4:

[0431] The terminal receives the rendered virtual reality space data and provides it to the user via a VR headset. The user wears the VR headset and virtually experiences an early 20th century factory. The input is the rendered virtual reality space data, and the output is a virtual environment experienced by the user visually and aurally.

[0432] Step 5:

[0433] The user interacts with the virtual reality space. For example, the user asks, "How does this machine work?" The device acquires the user's voice data and sends it to the server. The input is the user's voice data, and the output is the transmission of the voice data to the server.

[0434] Step 6:

[0435] The server uses an emotion recognition sensor to recognize emotions from the user's voice data. The input is voice data, and the output is emotion data (e.g., excitement, curiosity). The server analyzes this emotion data with an emotion engine.

[0436] Step 7:

[0437] The server uses a generative AI model to generate real-time responses based on the user's emotions, based on the emotion data acquired by the emotion engine. The input is the emotion data and the user's voice question, and the output is an appropriate response (e.g., "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time").

[0438] Step 8:

[0439] The terminal receives response data from the server and responds to the user in real time via the robot guide. The input is the response data, and the output is a voice response to the user. Specifically, the robot guide says, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[0440] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0441] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0442] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0443] [Second embodiment]

[0444] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0445] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0446] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0447] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0448] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0449] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0450] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0451] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0452] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0453] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0454] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0455] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0456] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures. The system of the present invention is implemented in the following manner.

[0457] System Overview

[0458] The system receives requests from users for specific eras or events, uses generative artificial intelligence to generate a virtual reality space based on the request, simulates historical figures within the virtual reality space, and provides the virtual reality space to a device for user interaction. It then generates real-time responses to the user's actual actions within the virtual reality space.

[0459] Program processing

[0460] 1. Receiving a user request

[0461] The terminal receives a request from the user selecting a past era or event and transmits it to the server.

[0462] Example: A user requests Japan during the Sengoku period.

[0463] 2. Creation of virtual reality space

[0464] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[0465] Example: The server generates backgrounds, scenery, and buildings for Japan during the Sengoku period.

[0466] 3. Simulation of historical figures

[0467] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[0468] Example: The server specifies the samurai's behavior patterns and conversation content.

[0469] 4. Provision of virtual reality space

[0470] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[0471] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[0472] 5. Generating real-time responses

[0473] The server generates responses from historical figures in real time based on the user's actions.

[0474] Example: When a user speaks to a samurai, the server uses generative artificial intelligence to generate the samurai's response and sends it to the device.

[0475] Specific examples

[0476] Example 1: Interacting with a samurai in Sengoku period Japan

[0477] 1. User Request

[0478] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[0479] 2. Creation of virtual reality space

[0480] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[0481] 3. Simulation of historical figures

[0482] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[0483] 4. Provision of virtual reality space

[0484] The device provides the user with a realistic virtual reality space via a VR headset.

[0485] 5. Interaction and real-time response

[0486] When the user talks to the samurai, the server generates the samurai's response in real time and sends it to the terminal. The samurai will respond with something like, "Can you tell me about your experience in today's battle?"

[0487] In this way, users can realistically experience past eras and enjoy conversations with historical figures. The present invention provides a system that allows users to easily experience visiting past eras.

[0488] The processing flow will be explained below.

[0489] Step 1:

[0490] The user launches an application.

[0491] Action: The device confirms that the application is running and displays the login screen.

[0492] Step 2:

[0493] The user enters their login information and logs in.

[0494] Operation: The device sends the user's login information to the server and performs the authentication process.

[0495] Step 3:

[0496] The user selects a particular era or event.

[0497] How it works: The device receives the user's selection through the UI and sends a request to the server.

[0498] Step 4:

[0499] The server receives and parses the request.

[0500] How it works: The server analyzes the user's request and identifies the required data.

[0501] Step 5:

[0502] The server activates generative artificial intelligence and generates a virtual reality space.

[0503] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[0504] Step 6:

[0505] The server formats the virtual reality space and converts it into a VR-compatible data format.

[0506] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[0507] Step 7:

[0508] The server generates data on historical figures.

[0509] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[0510] Step 8:

[0511] The server generates character dialogue scripts using generative artificial intelligence.

[0512] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[0513] Step 9:

[0514] The server sends the VR data to the device.

[0515] Operation: The server sends the generated VR space data to the device.

[0516] Step 10:

[0517] The device transfers the data to the VR headset.

[0518] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[0519] Step 11:

[0520] The user interacts with the VR headset and controllers.

[0521] Actions: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with characters.

[0522] Step 12:

[0523] The device detects the user's actions and sends them to the server.

[0524] How it works: The device detects the user's actions (e.g. talking to the samurai) and sends that information to the server.

[0525] Step 13:

[0526] The server analyzes the user's behavior and generates an appropriate response.

[0527] How it works: The server analyzes the user's behavior and uses generative artificial intelligence to generate real-time responses from historical figures.

[0528] Step 14:

[0529] The server sends the generated response data to the terminal.

[0530] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[0531] Step 15:

[0532] The terminal displays the response data to the user.

[0533] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[0534] Example 1

[0535] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] Conventional virtual reality systems, when users experience past eras or historical events, have problems with insufficient real-time interaction and response, and limited dialogue and behavior with historical characters. Furthermore, there is a lack of technology to generate highly accurate virtual reality spaces based on user requests. To address these issues, a system that can achieve real-time response and precise simulation by utilizing generative artificial intelligence is needed.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0538] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, means for simulating the behavior and conversation of historical figures in the virtual reality space, means for providing the virtual reality space to the user and transmitting the virtual reality space to a device with which the user interacts, means for monitoring interaction data from the user and generating responses in real time, means for generating 3D models of historical backgrounds, scenery, and buildings using generative artificial intelligence, and means for transmitting data of the generated virtual reality space to the user's device, thereby enabling the user to realistically experience past eras and enjoy conversations with historical figures.

[0539] "User" refers to any individual or entity that uses the System.

[0540] A "request" is a request in the form of a user inputting into a terminal their intention to experience a specific era or event.

[0541] "Terminal" refers to the device through which a user accesses the system, inputs requests, and experiences the virtual reality space, including, but not limited to, a smartphone, PC, or tablet.

[0542] "Server" refers to a central computer or cloud service that receives requests from users, uses generative artificial intelligence to simulate virtual reality spaces and historical figures, and sends the results to terminals.

[0543] "Generative AI" refers to AI technology that generates the actions and conversations of past eras and historical figures based on user requests. Specifically, it includes machine learning models and natural language processing models.

[0544] "Virtual reality space" refers to a 3D computer graphics environment created using generative artificial intelligence that recreates a specific era or event.

[0545] "Historical figures" refer to characters who actually existed in the past or who are spoken of as legends, and within this system their actions and conversations are simulated by generative artificial intelligence.

[0546] "Behavior" refers to the actions and movements of historical figures within the virtual reality space.

[0547] "Conversation" refers to a dialogue between a historical figure and the user or another character within a virtual reality space.

[0548] "Simulation" refers to the process of realistically recreating the behavior and conversations of historical figures in a virtual reality space using generative artificial intelligence.

[0549] "Interaction" refers to the operations and actions performed by the user within the virtual reality space, including the system's response to them.

[0550] "Real-time responses" refer to the responses and actions of historical figures that are generated instantly in response to user interactions.

[0551] "3D model" refers to a three-dimensional computer graphics object created by generative artificial intelligence, and specifically includes backgrounds, landscapes, buildings, etc.

[0552] The system of the present invention allows users to realistically experience past eras and enjoy direct communication with historical figures. Specific embodiments for carrying out the present invention will now be described in detail.

[0553] First, a user inputs a request to experience a specific era or event into a terminal. The terminal can be a smartphone, PC, tablet, or other device, and sends the user's input as an HTTP request to a server. The server then receives this request and generates a virtual reality space using generative artificial intelligence. Generative artificial intelligence uses machine learning models and natural language processing models, such as OpenAI's GPT model.

[0554] Based on user requests, the server retrieves information on historical background, landscapes, buildings, and other aspects, and performs advanced 3D modeling. Game engines such as Unity and Unreal Engine are used for this. The server also uses generative artificial intelligence to simulate the behavior and conversation of historical figures in the virtual reality space. The simulation includes characters such as samurai, feudal lords, and farmers.

[0555] The generated virtual reality space data is sent from the server to the user's device. The user wears a VR headset (e.g., Oculus Rift or HTC Vive) and experiences the virtual reality space. The data displayed on the VR headset includes Japanese landscapes, castles, and feudal lord mansions from the Warring States period, and the user can interact with them using the VR controller.

[0556] When a user performs a specific action in the virtual reality space, that data is sent to the server in real time, and the server uses generative artificial intelligence to generate an appropriate response. For example, when a user speaks to a samurai, the server generates the samurai's response and sends it to the device. If the user asks the samurai, "Will you tell me about your experience in today's battle?", the samurai can instantly generate and display a response such as, "I'll tell you about my experience in today's battle."

[0557] Specific examples

[0558] Example 1: Interacting with a samurai in Sengoku period Japan

[0559] The user inputs a request into the device saying, "I want to interact with a samurai in Sengoku-period Japan." This request is sent to the server as an HTTP request. The server uses generative artificial intelligence to generate Sengoku-period scenery, castles, and feudal lord mansions. The generated virtual reality space data is sent to the device, and the user experiences the virtual reality space using a VR headset. When the user asks the samurai, "Will you tell me about your experience in today's battle?", the server uses generative artificial intelligence to generate a response from the samurai, and the response "I'll tell you about my experience in today's battle" is displayed.

[0560] Prompt Sentence Examples

[0561] "I want to talk to a samurai from the Sengoku period. I want to learn his techniques."

[0562] "I would like to see a feudal lord's mansion. I would like to know what kind of decorations and furniture were there."

[0563] "I want to experience a day in the life of a farmer. I want to see what kind of work they do."

[0564] In this way, users can experience a past era in a realistic way and enjoy conversations with historical figures. This invention allows users to easily experience visiting a past era.

[0565] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0566] Step 1:

[0567] Receiving a user request

[0568] Input: The user types a request into the terminal to select a particular era or event.

[0569] Specific operation: The user inputs a request saying, "I want to interact with a samurai in Japan during the Warring States period."

[0570] Output: The terminal sends the user's request to the server as an HTTP request.

[0571] Step 2:

[0572] Processing virtual reality space generation requests

[0573] Input: The user request sent from the terminal.

[0574] Specific operation: The server receives the request and constructs a prompt to generate the virtual reality space using generative artificial intelligence (e.g., OpenAI's GPT model).

[0575] Output: Constructing prompts for generative artificial intelligence.

[0576] Step 3:

[0577] Virtual reality space generation

[0578] Input: A prompt for generative artificial intelligence.

[0579] How it works: The server generates 3D models of Japanese landscapes, castles, feudal lord mansions, etc. from the Warring States period based on prompts, using game engines such as Unity or Unreal Engine.

[0580] Output: 3D model data of the generated virtual reality space.

[0581] Step 4:

[0582] Simulation of historical figures

[0583] Input: 3D model data of the virtual reality space and additional prompt data.

[0584] Specific operation: The server uses generative artificial intelligence to generate the behavior and conversation scripts of historical figures in the virtual reality space. It generates the behavior patterns and conversation content of characters such as samurai, feudal lords, and farmers.

[0585] Output: Simulation data of the generated historical figures.

[0586] Step 5:

[0587] Integration and provision of virtual reality spaces

[0588] Input: 3D model data of virtual reality space and simulation data of historical figures.

[0589] How it works: The server aggregates the virtual reality data and sends it to the user's device, which then provides the virtual reality space to the user via a VR headset.

[0590] Output: The virtual reality space is displayed on the user's device.

[0591] Step 6:

[0592] Monitor interactions and generate real-time responses

[0593] Input: User behavior data (e.g., voice, movement).

[0594] How it works: The device sends user behavior data to the server, which then uses generative artificial intelligence to generate real-time responses based on the user's behavior. For example, when the user speaks to the samurai, the voice data is sent to the server, which then generates an appropriate response from the samurai.

[0595] Output: The generated real-time response data is sent to the terminal and displayed as an appropriate character's response in the virtual reality space.

[0596] As a specific example, if a user requests, "I want to interact with a samurai in Japan during the Warring States period," the user will go through the above steps to be immersed in a virtual reality space of the Warring States period and be able to enjoy real-time conversations with historical figures. This system makes it easy to experience visiting a past era.

[0597] (Application example 1)

[0598] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0599] Conventional virtual reality experience systems lacked the technology to enable real-time interaction with past eras or historical figures. In particular, it was difficult to generate intuitive and natural responses to user actions and interactions, limiting their ability to provide a realistic experience. Furthermore, there was no easy way to achieve such an experience using mobile devices such as smartphones. This resulted in a lack of interactivity in historical learning and entertainment.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0601] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era based on the request using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This enables a means for experiencing the virtual reality space using a smartphone and providing an interface for interacting with characters, and means for analyzing the conversation between the historical figure and the user using a natural language processing model and generating an appropriate response based on the content of the conversation.

[0602] definition statement

[0603] The "means for receiving a request from a user for a specific era or event" refers to a means for receiving the request within the system through an interface for a user to input a request for a past era or a specific event.

[0604] "Generative AI" is AI that has the ability to generate new data based on large amounts of data according to specified parameters.

[0605] A "means for generating a virtual reality space" is a means for recreating a past era on a computer based on a received request, and constructing a virtual environment that can be used visually and experientially by the user.

[0606] "Means for simulating the behavior and conversation of historical figures" refers to a means of imitating the behavior patterns and speaking styles of people who lived in past eras using generative artificial intelligence, enabling interaction with users.

[0607] "Devices for user interaction" are devices or interfaces that allow users to access a virtual reality space and perform operations and interactions within that space.

[0608] "Means for generating responses in real time" refers to a means for instantly generating appropriate responses to the user's actions and statements, and replying as a historical figure within the virtual reality space.

[0609] "Means for using a smartphone to experience a virtual reality space and provide an interface for interacting with characters" refers to a means for displaying a virtual reality space using a smartphone as a medium and for users to interact with characters within that space.

[0610] A "natural language processing model" is an artificial intelligence-based model for analyzing human language and understanding and generating its meaning and context.

[0611] "Means for generating appropriate responses" refers to techniques for providing natural and relevant replies based on the content of the dialogue with the user.

[0612] System configuration

[0613] This invention is composed of a user, a server, and a terminal. The user operates the system using a smartphone to request a specific era or event.

[0614] Program processing

[0615] 1. Receiving a user request

[0616] Users use a smartphone interface to request a specific time period or event from the past, which is then sent over the internet to a server.

[0617] 2. Creation of virtual reality space

[0618] The server uses OpenAI's generative AI model (e.g., GPT-3) to generate a virtual reality space based on the user's request, faithfully recreating historical and cultural elements.

[0619] 3. Simulation of historical figures

[0620] The server also uses generative AI models to simulate the behavior and conversation of historical figures, referencing a database of past events to ensure the simulated characters act and speak in a way appropriate to the time period.

[0621] 4. Provision of virtual reality space

[0622] The virtual reality space and simulated historical figures generated by the server are sent to the user's smartphone, where the user can access the virtual reality space and experience it using devices such as a VR headset and controller.

[0623] 5. Generating real-time responses

[0624] The server generates real-time responses to user interactions. When a user speaks to a historical figure, the content is analyzed using a natural language processing model, and an appropriate response is returned.

[0625] Hardware and Software

[0626] Hardware: Smartphone (iOS or Android compatible), VR headset

[0627] software:

[0628] OpenAI GPT-3 API: Used as a generative AI model to generate historical context and character behavior.

[0629] Unity3D: A platform and engine for building VR environments.

[0630] REST API: An interface for data communication between the server and the app.

[0631] Specific examples

[0632] If the user requests to interact with a warlord in Sengoku-era Japan, the following prompt sentence is used:

[0633] I would like to talk to warlords in Japan during the Sengoku period and learn about their daily lives and battles.

[0634] Based on this request, the system recreates the background, scenery, castles, and mansions of the Sengoku period, and generates the behavioral patterns and conversations of the warlords. When the user enters the virtual reality space using a VR headset and talks to the warlord, the warlord will respond in real time with a response such as, "Today was a tough battle. I fought and risked my life in it. It's time to show true courage."

[0635] This invention provides a system that allows users to easily experience visiting past eras in a realistic way using a smartphone and a generative AI model.

[0636] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0637] Program processing steps

[0638] Step 1:

[0639] Receiving a user request

[0640] explanation

[0641] Users launch the smartphone application and input the past era or event they wish to experience. This input is done by sending a request to the application screen, such as "I want to experience Japan during the Warring States period." The request is then sent to the server through the application interface.

[0642] Input and Output

[0643] Input: A request typed by a user through a smartphone interface (e.g., "I want to experience Japan during the Warring States period").

[0644] Output: The request data sent to the server.

[0645] operation

[0646] Specific operation: The user enters the desired era and event in the input form and presses the "Submit" button. The application sends the request data to the server.

[0647] Step 2:

[0648] Virtual reality space generation

[0649] explanation

[0650] The server processes the received request and sends a prompt to OpenAI's generative AI model (GPT-3). The prompt conveys details of the requested era and event to the generative AI model. The generative AI model generates background data for the virtual reality space based on the prompt.

[0651] Input and Output

[0652] Input: The request data received by the server and a prompt (e.g., "Recreate the setting and key events of Japan during the Warring States period.").

[0653] Output: Background data of the generated virtual reality space.

[0654] operation

[0655] How it works: The server takes in the request data, generates a prompt based on it, and sends it to the OpenAI API. The generative AI model generates background data for the virtual reality space in response to the prompt and returns it to the server.

[0656] Step 3:

[0657] Simulation of historical figures

[0658] explanation

[0659] The server uses a generative AI model to generate behavior and conversation data for historical figures, which involves referencing a historical database to set behavior patterns and conversation content appropriate for that era.

[0660] Input and Output

[0661] Input: Background data of the virtual reality space generated by the server and a prompt (e.g., "Generate the behavior patterns and speech patterns of a warlord from the Sengoku period.").

[0662] Output: Behavioral and conversational data of the generated historical figures.

[0663] operation

[0664] Specific operation: The server generates a more detailed prompt sentence and sends it to the generative AI model. The generative AI model generates behavior and conversation data of the historical figure based on the prompt and replies to the server.

[0665] Step 4:

[0666] Providing virtual reality space

[0667] explanation

[0668] The server sends the generated virtual reality space and simulation data to the user's smartphone, where the user can access the virtual reality space using an application.

[0669] Input and Output

[0670] Input: Generated virtual reality space and simulation data of historical figures.

[0671] Output: VR data sent to the user's smartphone.

[0672] operation

[0673] Specific operation: The server packages the generated virtual reality space data and sends it to the user's smartphone using a REST API. The user then accesses the virtual reality space through the smartphone screen and begins the experience.

[0674] Step 5:

[0675] Generate real-time responses

[0676] explanation

[0677] When a user interacts with a historical figure in the virtual reality space, the content of the interaction is transmitted to the server, which then analyzes the interaction using a natural language processing model and generates an appropriate response. The generated response is then transmitted in real time to the user's smartphone and displayed in the virtual reality space.

[0678] Input and Output

[0679] Input: User interaction (e.g., "How was your battle today?").

[0680] Output: The generated response of the historical figure (e.g., "Today was a tough battle. I risked my life in it. It's time to show true courage.").

[0681] operation

[0682] How it works: When a user speaks through the microphone in their VR headset, the voice data is sent to the server. The server uses a natural language processing model to convert the voice data into text, analyzes the text, and sends it to the generative AI model. The generative AI model generates a response text and sends it back to the server. The server then sends this response to the user's smartphone and displays it in the virtual reality space.

[0683] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0684] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures, and further relates to a system that provides a more natural and realistic experience by combining it with an emotion engine that recognizes the user's emotions. The system of the present invention is implemented in the following form.

[0685] System Overview

[0686] This system receives requests from users for specific eras or events, and uses generative artificial intelligence to generate a virtual reality space based on the request. It then simulates historical figures within the virtual reality space and provides the virtual reality space to a device for user interaction. It also uses an emotion engine to recognize the user's emotions, and adjusts the responses and behavior of the historical figures in the virtual reality space in real time based on the recognition results, providing a more emotionally relevant experience for the user.

[0687] Program processing

[0688] 1. Receiving a user request

[0689] The user selects a past era or event and sends a request from the terminal to the server.

[0690] Example: A user requests Japan during the Sengoku period.

[0691] 2. Creation of virtual reality space

[0692] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[0693] Example: The server generates the scenery, buildings and backgrounds of Japan during the Sengoku period.

[0694] 3. Simulation of historical figures

[0695] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[0696] Example: The server sets the samurai's behavior patterns and conversation content.

[0697] 4. Provision of virtual reality space

[0698] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[0699] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[0700] 5. Emotion Recognition and Analysis

[0701] The device uses an emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data.

[0702] Example: The device recognizes emotions such as joy, surprise, and sadness from the user's tone of voice and facial expressions.

[0703] 6. Generating real-time responses

[0704] The server generates real-time responses of historical figures based on the user's behavioral and emotional data.

[0705] Example: The user asks the samurai a question, and the samurai responds kindly according to the user's emotions.

[0706] Specific examples

[0707] Example 1: Interacting with a samurai in Sengoku period Japan

[0708] 1. User Request

[0709] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[0710] 2. Creation of virtual reality space

[0711] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[0712] 3. Simulation of historical figures

[0713] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[0714] 4. Provision of virtual reality space

[0715] The device provides the user with a realistic virtual reality space via a VR headset.

[0716] 5. Emotion Recognition and Analysis

[0717] The user asks the samurai a question, and the device uses an emotion engine to analyze the tone of voice and facial expressions to obtain emotional data.

[0718] 6. Interaction and real-time response

[0719] The server generates the samurai's response in real time based on the user's emotional data and provides it to the user via the terminal.

[0720] Example: If a user asks the samurai with a smile, "How can I become stronger?", the samurai will reply, "Your enthusiasm is already proof of your strength."

[0721] In this way, users can not only experience a past era realistically and converse with historical figures, but also enjoy a more emotionally responsive and realistic experience by adjusting their responses and behavior according to the user's emotions. The present invention realizes a system that allows users to easily experience visiting a past era while providing an emotionally rich experience.

[0722] The processing flow will be explained below.

[0723] Step 1:

[0724] The user launches an application.

[0725] Action: The device confirms that the application is running and displays the login screen.

[0726] Step 2:

[0727] The user enters their login information and logs in.

[0728] Operation: The device sends the user's login information to the server and performs the authentication process.

[0729] Step 3:

[0730] The user selects a particular era or event.

[0731] How it works: The device receives the user's selection through the UI and sends a request to the server.

[0732] Step 4:

[0733] The server receives and parses the request.

[0734] How it works: The server analyzes the user's request and identifies the required data.

[0735] Step 5:

[0736] The server activates generative artificial intelligence and generates a virtual reality space.

[0737] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[0738] Step 6:

[0739] The server formats the virtual reality space and converts it into a VR-compatible data format.

[0740] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[0741] Step 7:

[0742] The server generates data on historical figures.

[0743] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[0744] Step 8:

[0745] The server generates character dialogue scripts using generative artificial intelligence.

[0746] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[0747] Step 9:

[0748] The server sends the VR data to the device.

[0749] Operation: The server sends the generated VR space data to the device.

[0750] Step 10:

[0751] The device transfers the data to the VR headset.

[0752] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[0753] Step 11:

[0754] The user interacts with the VR headset and controllers.

[0755] How it works: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with the characters.

[0756] Step 12:

[0757] The device analyzes the user's voice and facial expressions using an emotion engine.

[0758] How it works: The device analyzes emotional data from the user's tone of voice and facial expressions and sends that data to the server.

[0759] Step 13:

[0760] The server analyzes the user's behavioral and emotional data.

[0761] How it works: The server comprehensively analyzes the user's behavior and emotional data to determine the appropriate response of the historical figure.

[0762] Step 14:

[0763] The server sends the generated response data to the terminal.

[0764] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[0765] Step 15:

[0766] The terminal displays the response data to the user.

[0767] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[0768] Example 2

[0769] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0770] In conventional virtual reality systems, when users experience past eras, dialogue and interactions with historical figures cannot be adjusted in real time, resulting in a lack of realism and immersion. Furthermore, there was no system that could generate responses based on the user's emotional state, making it difficult to provide an experience that is more in tune with each individual user's emotions. Furthermore, the technical challenge of analyzing emotional data such as the user's voice and facial expressions and generating corresponding responses in real time remained unresolved.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This allows the user to realistically experience a past era, adjust dialogue and interaction with historical figures in real time, and generate responses according to the user's emotional state.

[0772] The "user request receiving means" is a means for the user to select a specific era or event and transmit that information to the system.

[0773] "Generative AI" is an AI technology used to recreate past eras, and is an AI capable of generating virtual reality spaces based on requests.

[0774] A "virtual reality space generation means" is a means for constructing a virtual reality space that recreates a specific era or event using generative artificial intelligence.

[0775] A "means for simulating historical figures" is a means for simulating the behavior and conversation of historical figures in a virtual reality space, providing users with realistic interactions.

[0776] The "virtual reality space providing means" refers to a device or method that provides the generated virtual reality space to the user and allows the user to interact within that space.

[0777] The "emotion engine" is an engine that analyzes the user's voice, facial expressions, body movements, etc., and recognizes their emotional state.

[0778] The "emotion data acquisition means" is a means for acquiring the user's emotion data analyzed using the emotion engine.

[0779] The "real-time response generation means" is a means for generating responses of characters in a virtual reality space in real time based on the user's behavior and emotional data.

[0780] The present invention relates to a system that allows users to experience past eras in a realistic way and enjoy direct communication with historical figures. Furthermore, the system provides a more natural and immersive experience by incorporating an emotion engine that recognizes the user's emotions. This system is implemented as follows.

[0781] System Overview

[0782] This system first receives a request for a specific era or event from the user. The user operates the terminal, selects a specific era or event from the past, and sends the request to the server. For example, a request might be, "I want to talk to a samurai in Japan during the Warring States period."

[0783] The server then uses generative artificial intelligence (e.g., GPT-4) to generate a virtual reality space based on the request. Specifically, it uses 3D rendering technology and generative AI models to create the scenery, buildings, and backgrounds of the requested era. For example, it generates a Japanese landscape, castle, or feudal lord's mansion from the Warring States period.

[0784] The server then generates behavior and conversation scripts for historical figures (such as samurai, feudal lords, and farmers) who will be active in the virtual reality space. This uses natural language processing technology to simulate realistic conversations. For example, it generates an object called a "samurai" and sets its behavioral patterns. The behavioral patterns include actions such as "patrolling the castle in the morning" and "talking when the user approaches."

[0785] The generated virtual reality space is provided to the user via a terminal. The user can move freely within the virtual space using a VR headset and controllers and interact with historical figures. For example, the user wears a VR headset and uses the controllers in their hands to walk and touch within the virtual space. The terminal renders the video and audio in real time.

[0786] The system also includes an emotion engine. The device uses the emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data. For example, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine then analyzes. The analyzed emotional data includes emotions such as joy, surprise, and sadness.

[0787] Finally, the server generates real-time responses from historical figures in the virtual reality space based on the user's behavioral and emotional data. For example, if the user shows a surprised expression, the samurai might respond, "Did I surprise you?" In this way, users can realistically experience past eras and feel a sense of presence when interacting with historical figures.

[0788] As a concrete example, consider the case where a user sends a request from a device saying, "I want to interact with a samurai in Sengoku-period Japan." In this case, the server generates the scenery, castles, and feudal lord mansions of Sengoku-period Japan, and sets up the actions and conversations of samurai, peasants, and feudal lord characters in the virtual reality space. The device then provides the user with a realistic virtual reality space via a VR headset, and when the user asks the samurai a question, the device obtains emotional data from the user's tone of voice and facial expressions, and generates the samurai's response in real time based on that data.

[0789] As a result, users can not only visit past eras, but also enjoy a more emotionally responsive and realistic experience, as responses and behaviors are adjusted according to the user's emotions.

[0790] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0791] Step 1:

[0792] Receiving a user request

[0793] The user selects a specific time period or event in the past and sends a request from the device to the server. The input is the time period or event selected by the user (e.g., "Japan during the Warring States period"). This input data is sent to the server via the device. Specifically, when the user makes a selection using the touchscreen and presses the confirmation button, the device forwards this to the server as an HTTP request. The output is data that the server has received and analyzed the request content.

[0794] Step 2:

[0795] Virtual reality space generation

[0796] The server analyzes the request and generates a virtual reality space using generative artificial intelligence (e.g., GPT-4). The input is information about a specific era or event received from the user. Based on this information, the server generates data for 3D rendering of landscapes, buildings, backgrounds, etc. Specifically, after analyzing the request, the server retrieves a dataset corresponding to "Japan during the Warring States period" and generates virtual castles and landscapes using a generative AI model. The output is data for the generated virtual reality space.

[0797] Step 3:

[0798] Simulation of historical figures

[0799] The server generates the behavior and conversation scripts of historical figures active within the virtual reality space. The inputs include data on the generated virtual reality space and information about the historical figures. Based on this, the server uses natural language processing technology to simulate realistic conversations. In terms of specific operations, the server generates an object called a "samurai" and sets its behavior patterns (e.g., "patrol the castle in the morning" or "talk to the user when they approach"). The output is the behavior patterns and conversation scripts of the generated historical figures.

[0800] Step 4:

[0801] Providing virtual reality space

[0802] The device provides the generated virtual reality space to the user. The input is virtual reality space data sent from the server. The device transfers this data to a VR headset and controller, allowing the user to move freely within the virtual space and interact with historical figures. Specifically, the user wears a VR headset and uses the controller in their hand to walk and touch within the virtual space. The output is the virtual reality space that the user actually experiences.

[0803] Step 5:

[0804] Emotion Recognition and Analysis

[0805] The device analyzes the user's voice, facial expressions, and body movements using an emotion engine to obtain emotion data. The input data includes the user's voice, facial expressions, and body movements. Specifically, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine analyzes (e.g., if the user smiles, it is recognized as "joy"). The output is the analyzed emotion data.

[0806] Step 6:

[0807] Generate real-time responses

[0808] The server generates a real-time response from the historical figure based on the user's behavior and emotion data. The input is the user's behavior data and emotion data. Specifically, the server receives the data sent from the emotion engine and adjusts the content of the historical figure's remarks (for example, if the user shows a surprised expression, the samurai will respond, "Did I surprise you?"). The output is the historical figure's real-time response.

[0809] This allows users to not only experience past eras in a realistic way and interact with historical figures, but also enjoy a more emotionally immersive experience as their responses and behavior are adjusted according to the user's emotions.

[0810] (Application example 2)

[0811] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0812] Conventional virtual reality systems lack emotional responses and interactions when users experience past times, making the experience less immersive. Additionally, factories and other historical sites tend to rely solely on modern technology and data to provide explanations, lacking in consideration of historical context and user emotions.

[0813] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0814] In this invention, the server includes means for receiving requests for specific eras or events from users, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures within the virtual reality space, thereby enabling users to experience a past era with a sense of realism that includes emotions.

[0815] The "means for receiving a request for a specific era or event from a user" is an interface or device that allows a user to select and request a specific era or event that the user wishes to experience.

[0816] "Generative AI" is an AI technology that automatically generates past eras and scenarios based on user requests.

[0817] "Means for generating virtual reality spaces" refers to devices or software that use generative artificial intelligence to create three-dimensional virtual spaces that faithfully recreate past eras and events.

[0818] "Means for simulating the behavior and conversation of historical figures within a virtual reality space" refers to devices or software for simulating in real time the movements and conversation of historical figures appearing within a virtual reality space.

[0819] "Means for providing a virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with" means means for providing the generated virtual reality space to a user and transmitting data to a headset or other device for the user to experience and interact within the space.

[0820] An "emotion engine" is software that analyzes and recognizes emotions from a user's voice, facial expressions, actions, etc.

[0821] The "means for generating responses of historical figures in real time based on emotion data" is a technology for generating responses and behaviors of historical figures in real time based on the emotion data of a user.

[0822] This invention relates to a virtual reality system that allows users to experience past eras realistically and interact with historical figures. By combining it with an emotion engine, it provides real-time responses and behaviors according to the user's emotions, realizing a more natural and immersive experience.

[0823] System Program Overview

[0824] The server receives requests from users for specific eras or events. For example, if a user requests a "factory tour from the early 20th century," the server analyzes the request and uses generative artificial intelligence to generate a virtual reality space. This virtual reality space realistically reproduces the factory's scenery, the operation of machines, and the behavior of workers.

[0825] The server then simulates the behavior and conversation of historical figures (in this case, factory workers and engineers) within the virtual reality space, enabling realistic interactions within the virtual reality space experienced by the user.

[0826] The generated virtual reality space is provided to the user via a VR headset and controller, allowing the user to experience a past era.

[0827] Hardware and Software Details

[0828] VR headset: A device that provides users with a virtual reality space.

[0829] Robot guide: A robot for interacting with users.

[0830] Emotion recognition sensor: A sensor that acquires emotions from the user's voice, facial expressions, and movements.

[0831] OpenAI GPT-3: As a generative artificial intelligence, it is used to analyze user requests and generate real-time responses based on virtual reality spatial data and emotions.

[0832] Emotion Recognition Library: A library for analyzing user emotions and obtaining emotional data.

[0833] VR Renderer: Rendering software for generating virtual reality spaces.

[0834] Robot Controller software: Software for controlling the operation of the robot guide.

[0835] Specific examples

[0836] For example, if a user requests "I want to experience a factory tour from the early 20th century," the server generates a virtual reality space using the following prompts:

[0837] "A user requests a tour of an early 20th century factory. What kind of scene would you generate?"

[0838] In the generated virtual reality space, old machinery and work scenes in the factory are realistically reproduced. If the user is excited, the emotion recognition sensor will recognize that emotion and the emotion engine will analyze the data. For example, if the user excitedly asks, "How does this machine work?", the robot guide can respond, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[0839] In this way, a system is realized that allows users to experience a past era with a sense of realism while engaging in emotional interaction.

[0840] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0841] Step 1:

[0842] The user inputs a request to experience a specific era or event. Specifically, the user sends a request via their terminal saying, "I would like to experience a factory tour from the early 20th century." This input data is sent to the server, which receives and analyzes the request.

[0843] Step 2:

[0844] Based on the received request, the server generates prompts to generate a virtual reality space using a generative AI model (OpenAI GPT-3). The input is the user's request, and the output is virtual data that defines the specific scenario and components of the virtual reality space. For example, it generates a prompt such as, "The user has requested a factory tour from the early 20th century. What kind of scene would you like to generate?"

[0845] Step 3:

[0846] The server inputs prompts into the generative AI model to generate details of the virtual reality space. The input is the prompt sentence, and the output is detailed virtual reality space data (e.g., factory scenery, machine operation, worker behavior, etc.). The server passes this output to the VR Renderer, which renders the virtual reality space.

[0847] Step 4:

[0848] The terminal receives the rendered virtual reality space data and provides it to the user via a VR headset. The user wears the VR headset and virtually experiences an early 20th century factory. The input is the rendered virtual reality space data, and the output is a virtual environment experienced by the user visually and aurally.

[0849] Step 5:

[0850] The user interacts with the virtual reality space. For example, the user asks, "How does this machine work?" The device acquires the user's voice data and sends it to the server. The input is the user's voice data, and the output is the transmission of the voice data to the server.

[0851] Step 6:

[0852] The server uses an emotion recognition sensor to recognize emotions from the user's voice data. The input is voice data, and the output is emotion data (e.g., excitement, curiosity). The server analyzes this emotion data with an emotion engine.

[0853] Step 7:

[0854] The server uses a generative AI model to generate real-time responses based on the user's emotions, based on the emotion data acquired by the emotion engine. The input is the emotion data and the user's voice question, and the output is an appropriate response (e.g., "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time").

[0855] Step 8:

[0856] The terminal receives response data from the server and responds to the user in real time via the robot guide. The input is the response data, and the output is a voice response to the user. Specifically, the robot guide says, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[0857] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0858] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0859] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0860] [Third embodiment]

[0861] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0862] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0863] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0864] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0865] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0866] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0867] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0868] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0869] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0870] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0871] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0872] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0873] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures. The system of the present invention is implemented in the following manner.

[0874] System Overview

[0875] The system receives requests from users for specific eras or events, uses generative artificial intelligence to generate a virtual reality space based on the request, simulates historical figures within the virtual reality space, and provides the virtual reality space to a device for user interaction. It then generates real-time responses to the user's actual actions within the virtual reality space.

[0876] Program processing

[0877] 1. Receiving a user request

[0878] The terminal receives a request from the user selecting a past era or event and transmits it to the server.

[0879] Example: A user requests Japan during the Sengoku period.

[0880] 2. Creation of virtual reality space

[0881] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[0882] Example: The server generates backgrounds, scenery, and buildings for Japan during the Sengoku period.

[0883] 3. Simulation of historical figures

[0884] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[0885] Example: The server specifies the samurai's behavior patterns and conversation content.

[0886] 4. Provision of virtual reality space

[0887] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[0888] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[0889] 5. Generating real-time responses

[0890] The server generates responses from historical figures in real time based on the user's actions.

[0891] Example: When a user speaks to a samurai, the server uses generative artificial intelligence to generate the samurai's response and sends it to the device.

[0892] Specific examples

[0893] Example 1: Interacting with a samurai in Sengoku period Japan

[0894] 1. User Request

[0895] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[0896] 2. Creation of virtual reality space

[0897] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[0898] 3. Simulation of historical figures

[0899] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[0900] 4. Provision of virtual reality space

[0901] The device provides the user with a realistic virtual reality space via a VR headset.

[0902] 5. Interaction and real-time response

[0903] When the user talks to the samurai, the server generates the samurai's response in real time and sends it to the terminal. The samurai will respond with something like, "Can you tell me about your experience in today's battle?"

[0904] In this way, users can realistically experience past eras and enjoy conversations with historical figures. The present invention provides a system that allows users to easily experience visiting past eras.

[0905] The processing flow will be explained below.

[0906] Step 1:

[0907] The user launches an application.

[0908] Action: The device confirms that the application is running and displays the login screen.

[0909] Step 2:

[0910] The user enters their login information and logs in.

[0911] Operation: The device sends the user's login information to the server and performs the authentication process.

[0912] Step 3:

[0913] The user selects a particular era or event.

[0914] How it works: The device receives the user's selection through the UI and sends a request to the server.

[0915] Step 4:

[0916] The server receives and parses the request.

[0917] How it works: The server analyzes the user's request and identifies the required data.

[0918] Step 5:

[0919] The server activates generative artificial intelligence and generates a virtual reality space.

[0920] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[0921] Step 6:

[0922] The server formats the virtual reality space and converts it into a VR-compatible data format.

[0923] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[0924] Step 7:

[0925] The server generates data on historical figures.

[0926] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[0927] Step 8:

[0928] The server generates character dialogue scripts using generative artificial intelligence.

[0929] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[0930] Step 9:

[0931] The server sends the VR data to the device.

[0932] Operation: The server sends the generated VR space data to the device.

[0933] Step 10:

[0934] The device transfers the data to the VR headset.

[0935] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[0936] Step 11:

[0937] The user interacts with the VR headset and controllers.

[0938] Actions: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with characters.

[0939] Step 12:

[0940] The device detects the user's actions and sends them to the server.

[0941] How it works: The device detects the user's actions (e.g. talking to the samurai) and sends that information to the server.

[0942] Step 13:

[0943] The server analyzes the user's behavior and generates an appropriate response.

[0944] How it works: The server analyzes the user's behavior and uses generative artificial intelligence to generate real-time responses from historical figures.

[0945] Step 14:

[0946] The server sends the generated response data to the terminal.

[0947] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[0948] Step 15:

[0949] The terminal displays the response data to the user.

[0950] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[0951] Example 1

[0952] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0953] Conventional virtual reality systems, when users experience past eras or historical events, have problems with insufficient real-time interaction and response, and limited dialogue and behavior with historical characters. Furthermore, there is a lack of technology to generate highly accurate virtual reality spaces based on user requests. To address these issues, a system that can achieve real-time response and precise simulation by utilizing generative artificial intelligence is needed.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0955] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, means for simulating the behavior and conversation of historical figures in the virtual reality space, means for providing the virtual reality space to the user and transmitting the virtual reality space to a device with which the user interacts, means for monitoring interaction data from the user and generating responses in real time, means for generating 3D models of historical backgrounds, scenery, and buildings using generative artificial intelligence, and means for transmitting data of the generated virtual reality space to the user's device, thereby enabling the user to realistically experience past eras and enjoy conversations with historical figures.

[0956] "User" refers to any individual or entity that uses the System.

[0957] A "request" is a request in the form of a user inputting into a terminal their intention to experience a specific era or event.

[0958] "Terminal" refers to the device through which a user accesses the system, inputs requests, and experiences the virtual reality space, including, but not limited to, a smartphone, PC, or tablet.

[0959] "Server" refers to a central computer or cloud service that receives requests from users, uses generative artificial intelligence to simulate virtual reality spaces and historical figures, and sends the results to terminals.

[0960] "Generative AI" refers to AI technology that generates the actions and conversations of past eras and historical figures based on user requests. Specifically, it includes machine learning models and natural language processing models.

[0961] "Virtual reality space" refers to a 3D computer graphics environment created using generative artificial intelligence that recreates a specific era or event.

[0962] "Historical figures" refer to characters who actually existed in the past or who are spoken of as legends, and within this system their actions and conversations are simulated by generative artificial intelligence.

[0963] "Behavior" refers to the actions and movements of historical figures within the virtual reality space.

[0964] "Conversation" refers to a dialogue between a historical figure and the user or another character within a virtual reality space.

[0965] "Simulation" refers to the process of realistically recreating the behavior and conversations of historical figures in a virtual reality space using generative artificial intelligence.

[0966] "Interaction" refers to the operations and actions performed by the user within the virtual reality space, including the system's response to them.

[0967] "Real-time responses" refer to the responses and actions of historical figures that are generated instantly in response to user interactions.

[0968] "3D model" refers to a three-dimensional computer graphics object created by generative artificial intelligence, and specifically includes backgrounds, landscapes, buildings, etc.

[0969] The system of the present invention allows users to realistically experience past eras and enjoy direct communication with historical figures. Specific embodiments for carrying out the present invention will now be described in detail.

[0970] First, a user inputs a request to experience a specific era or event into a terminal. The terminal can be a smartphone, PC, tablet, or other device, and sends the user's input as an HTTP request to a server. The server then receives this request and generates a virtual reality space using generative artificial intelligence. Generative artificial intelligence uses machine learning models and natural language processing models, such as OpenAI's GPT model.

[0971] Based on user requests, the server retrieves information on historical background, landscapes, buildings, and other aspects, and performs advanced 3D modeling. Game engines such as Unity and Unreal Engine are used for this. The server also uses generative artificial intelligence to simulate the behavior and conversation of historical figures in the virtual reality space. The simulation includes characters such as samurai, feudal lords, and farmers.

[0972] The generated virtual reality space data is sent from the server to the user's device. The user wears a VR headset (e.g., Oculus Rift or HTC Vive) and experiences the virtual reality space. The data displayed on the VR headset includes Japanese landscapes, castles, and feudal lord mansions from the Warring States period, and the user can interact with them using the VR controller.

[0973] When a user performs a specific action in the virtual reality space, that data is sent to the server in real time, and the server uses generative artificial intelligence to generate an appropriate response. For example, when a user speaks to a samurai, the server generates the samurai's response and sends it to the device. If the user asks the samurai, "Will you tell me about your experience in today's battle?", the samurai can instantly generate and display a response such as, "I'll tell you about my experience in today's battle."

[0974] Specific examples

[0975] Example 1: Interacting with a samurai in Sengoku period Japan

[0976] The user inputs a request into the device saying, "I want to interact with a samurai in Sengoku-period Japan." This request is sent to the server as an HTTP request. The server uses generative artificial intelligence to generate Sengoku-period scenery, castles, and feudal lord mansions. The generated virtual reality space data is sent to the device, and the user experiences the virtual reality space using a VR headset. When the user asks the samurai, "Will you tell me about your experience in today's battle?", the server uses generative artificial intelligence to generate a response from the samurai, and the response "I'll tell you about my experience in today's battle" is displayed.

[0977] Prompt Sentence Examples

[0978] "I want to talk to a samurai from the Sengoku period. I want to learn his techniques."

[0979] "I would like to see a feudal lord's mansion. I would like to know what kind of decorations and furniture were there."

[0980] "I want to experience a day in the life of a farmer. I want to see what kind of work they do."

[0981] In this way, users can experience a past era in a realistic way and enjoy conversations with historical figures. This invention allows users to easily experience visiting a past era.

[0982] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0983] Step 1:

[0984] Receiving a user request

[0985] Input: The user types a request into the terminal to select a particular era or event.

[0986] Specific operation: The user inputs a request saying, "I want to interact with a samurai in Japan during the Warring States period."

[0987] Output: The terminal sends the user's request to the server as an HTTP request.

[0988] Step 2:

[0989] Processing virtual reality space generation requests

[0990] Input: The user request sent from the terminal.

[0991] Specific operation: The server receives the request and constructs a prompt to generate the virtual reality space using generative artificial intelligence (e.g., OpenAI's GPT model).

[0992] Output: Constructing prompts for generative artificial intelligence.

[0993] Step 3:

[0994] Virtual reality space generation

[0995] Input: A prompt for generative artificial intelligence.

[0996] How it works: The server generates 3D models of Japanese landscapes, castles, feudal lord mansions, etc. from the Warring States period based on prompts, using game engines such as Unity or Unreal Engine.

[0997] Output: 3D model data of the generated virtual reality space.

[0998] Step 4:

[0999] Simulation of historical figures

[1000] Input: 3D model data of the virtual reality space and additional prompt data.

[1001] Specific operation: The server uses generative artificial intelligence to generate the behavior and conversation scripts of historical figures in the virtual reality space. It generates the behavior patterns and conversation content of characters such as samurai, feudal lords, and farmers.

[1002] Output: Simulation data of the generated historical figures.

[1003] Step 5:

[1004] Integration and provision of virtual reality spaces

[1005] Input: 3D model data of virtual reality space and simulation data of historical figures.

[1006] How it works: The server aggregates the virtual reality data and sends it to the user's device, which then provides the virtual reality space to the user via a VR headset.

[1007] Output: The virtual reality space is displayed on the user's device.

[1008] Step 6:

[1009] Monitor interactions and generate real-time responses

[1010] Input: User behavior data (e.g., voice, movement).

[1011] How it works: The device sends user behavior data to the server, which then uses generative artificial intelligence to generate real-time responses based on the user's behavior. For example, when the user speaks to the samurai, the voice data is sent to the server, which then generates an appropriate response from the samurai.

[1012] Output: The generated real-time response data is sent to the terminal and displayed as an appropriate character's response in the virtual reality space.

[1013] As a specific example, if a user requests, "I want to interact with a samurai in Japan during the Warring States period," the user will go through the above steps to be immersed in a virtual reality space of the Warring States period and be able to enjoy real-time conversations with historical figures. This system makes it easy to experience visiting a past era.

[1014] (Application example 1)

[1015] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1016] Conventional virtual reality experience systems lacked the technology to enable real-time interaction with past eras or historical figures. In particular, it was difficult to generate intuitive and natural responses to user actions and interactions, limiting their ability to provide a realistic experience. Furthermore, there was no easy way to achieve such an experience using mobile devices such as smartphones. This resulted in a lack of interactivity in historical learning and entertainment.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1018] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era based on the request using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This enables a means for experiencing the virtual reality space using a smartphone and providing an interface for interacting with characters, and means for analyzing the conversation between the historical figure and the user using a natural language processing model and generating an appropriate response based on the content of the conversation.

[1019] definition statement

[1020] The "means for receiving a request from a user for a specific era or event" refers to a means for receiving the request within the system through an interface for a user to input a request for a past era or a specific event.

[1021] "Generative AI" is AI that has the ability to generate new data based on large amounts of data according to specified parameters.

[1022] A "means for generating a virtual reality space" is a means for recreating a past era on a computer based on a received request, and constructing a virtual environment that can be used visually and experientially by the user.

[1023] "Means for simulating the behavior and conversation of historical figures" refers to a means of imitating the behavior patterns and speaking styles of people who lived in past eras using generative artificial intelligence, enabling interaction with users.

[1024] "Devices for user interaction" are devices or interfaces that allow users to access a virtual reality space and perform operations and interactions within that space.

[1025] "Means for generating responses in real time" refers to a means for instantly generating appropriate responses to the user's actions and statements, and replying as a historical figure within the virtual reality space.

[1026] "Means for providing an interface for experiencing a virtual reality space using a smartphone and interacting with characters" refers to a means for displaying a virtual reality space using a smartphone as a medium and for users to interact with characters within that space.

[1027] A "natural language processing model" is an artificial intelligence-based model for analyzing human language and understanding and generating its meaning and context.

[1028] "Means for generating appropriate responses" refers to techniques for providing natural and relevant replies based on the content of the dialogue with the user.

[1029] System configuration

[1030] This invention is composed of a user, a server, and a terminal. The user operates the system using a smartphone to request a specific era or event.

[1031] Program processing

[1032] 1. Receiving a user request

[1033] Users use a smartphone interface to request a specific time period or event from the past, which is then sent over the internet to a server.

[1034] 2. Creation of virtual reality space

[1035] The server uses OpenAI's generative AI model (e.g., GPT-3) to generate a virtual reality space based on the user's request, faithfully recreating historical and cultural elements.

[1036] 3. Simulation of historical figures

[1037] The server also uses generative AI models to simulate the behavior and conversation of historical figures, referencing a database of past events to ensure the simulated characters act and speak in a way appropriate to the time period.

[1038] 4. Provision of virtual reality space

[1039] The virtual reality space and simulated historical figures generated by the server are sent to the user's smartphone, where the user can access the virtual reality space and experience it using devices such as a VR headset and controller.

[1040] 5. Generating real-time responses

[1041] The server generates real-time responses to user interactions. When a user speaks to a historical figure, the content is analyzed using a natural language processing model, and an appropriate response is returned.

[1042] Hardware and Software

[1043] Hardware: Smartphone (iOS or Android compatible), VR headset

[1044] software:

[1045] OpenAI GPT-3 API: Used as a generative AI model to generate historical context and character behavior.

[1046] Unity3D: A platform and engine for building VR environments.

[1047] REST API: An interface for data communication between the server and the app.

[1048] Specific examples

[1049] If the user requests to interact with a warlord in Sengoku-era Japan, the following prompt sentence is used:

[1050] I would like to talk to warlords in Japan during the Sengoku period and learn about their daily lives and battles.

[1051] Based on this request, the system recreates the background, scenery, castles, and mansions of the Sengoku period, and generates the behavioral patterns and conversations of the warlords. When the user enters the virtual reality space using a VR headset and talks to the warlord, the warlord will respond in real time with a response such as, "Today was a tough battle. I fought and risked my life in it. It's time to show true courage."

[1052] This invention provides a system that allows users to easily experience visiting past eras in a realistic way using a smartphone and a generative AI model.

[1053] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1054] Program processing steps

[1055] Step 1:

[1056] Receiving a user request

[1057] explanation

[1058] Users launch the smartphone application and input the past era or event they wish to experience. This input is done by sending a request to the application screen, such as "I want to experience Japan during the Warring States period." The request is then sent to the server through the application interface.

[1059] Input and Output

[1060] Input: A request typed by a user through a smartphone interface (e.g., "I want to experience Japan during the Warring States period").

[1061] Output: The request data sent to the server.

[1062] operation

[1063] Specific operation: The user enters the desired era and event in the input form and presses the "Submit" button. The application sends the request data to the server.

[1064] Step 2:

[1065] Virtual reality space generation

[1066] explanation

[1067] The server processes the received request and sends a prompt to OpenAI's generative AI model (GPT-3). The prompt conveys details of the requested era and event to the generative AI model. The generative AI model generates background data for the virtual reality space based on the prompt.

[1068] Input and Output

[1069] Input: The request data received by the server and a prompt (e.g., "Recreate the setting and key events of Japan during the Warring States period.").

[1070] Output: Background data of the generated virtual reality space.

[1071] operation

[1072] How it works: The server takes in the request data, generates a prompt based on it, and sends it to the OpenAI API. The generative AI model generates background data for the virtual reality space in response to the prompt and returns it to the server.

[1073] Step 3:

[1074] Simulation of historical figures

[1075] explanation

[1076] The server uses a generative AI model to generate behavior and conversation data for historical figures, which involves referencing a historical database to set behavior patterns and conversation content appropriate for that era.

[1077] Input and Output

[1078] Input: Background data of the virtual reality space generated by the server and a prompt (e.g., "Generate the behavior patterns and speech patterns of a warlord from the Sengoku period.").

[1079] Output: Behavioral and conversational data of the generated historical figures.

[1080] operation

[1081] Specific operation: The server generates a more detailed prompt sentence and sends it to the generative AI model. The generative AI model generates behavior and conversation data of the historical figure based on the prompt and replies to the server.

[1082] Step 4:

[1083] Providing virtual reality space

[1084] explanation

[1085] The server sends the generated virtual reality space and simulation data to the user's smartphone, where the user can access the virtual reality space using an application.

[1086] Input and Output

[1087] Input: Generated virtual reality space and simulation data of historical figures.

[1088] Output: VR data sent to the user's smartphone.

[1089] operation

[1090] Specific operation: The server packages the generated virtual reality space data and sends it to the user's smartphone using a REST API. The user then accesses the virtual reality space through the smartphone screen and begins the experience.

[1091] Step 5:

[1092] Generate real-time responses

[1093] explanation

[1094] When a user interacts with a historical figure in the virtual reality space, the content of the interaction is transmitted to the server, which then analyzes the interaction using a natural language processing model and generates an appropriate response. The generated response is then transmitted in real time to the user's smartphone and displayed in the virtual reality space.

[1095] Input and Output

[1096] Input: User interaction (e.g., "How was your battle today?").

[1097] Output: The generated response of the historical figure (e.g., "Today was a tough battle. I risked my life in it. It's time to show true courage.").

[1098] operation

[1099] How it works: When a user speaks through the microphone in their VR headset, the voice data is sent to the server. The server uses a natural language processing model to convert the voice data into text, analyzes the text, and sends it to the generative AI model. The generative AI model generates a response text and sends it back to the server. The server then sends this response to the user's smartphone and displays it in the virtual reality space.

[1100] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1101] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures, and further relates to a system that provides a more natural and realistic experience by combining it with an emotion engine that recognizes the user's emotions. The system of the present invention is implemented in the following form.

[1102] System Overview

[1103] This system receives requests from users for specific eras or events, and uses generative artificial intelligence to generate a virtual reality space based on the request. It then simulates historical figures within the virtual reality space and provides the virtual reality space to a device for user interaction. It also uses an emotion engine to recognize the user's emotions, and adjusts the responses and behavior of the historical figures in the virtual reality space in real time based on the recognition results, providing a more emotionally relevant experience for the user.

[1104] Program processing

[1105] 1. Receiving a user request

[1106] The user selects a past era or event and sends a request from the terminal to the server.

[1107] Example: A user requests Japan during the Sengoku period.

[1108] 2. Creation of virtual reality space

[1109] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[1110] Example: The server generates the scenery, buildings and backgrounds of Japan during the Sengoku period.

[1111] 3. Simulation of historical figures

[1112] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[1113] Example: The server sets the samurai's behavior patterns and conversation content.

[1114] 4. Provision of virtual reality space

[1115] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[1116] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[1117] 5. Emotion Recognition and Analysis

[1118] The device uses an emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data.

[1119] Example: The device recognizes emotions such as joy, surprise, and sadness from the user's tone of voice and facial expressions.

[1120] 6. Generating real-time responses

[1121] The server generates real-time responses of historical figures based on the user's behavioral and emotional data.

[1122] Example: The user asks the samurai a question, and the samurai responds kindly according to the user's emotions.

[1123] Specific examples

[1124] Example 1: Interacting with a samurai in Sengoku period Japan

[1125] 1. User Request

[1126] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[1127] 2. Creation of virtual reality space

[1128] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[1129] 3. Simulation of historical figures

[1130] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[1131] 4. Provision of virtual reality space

[1132] The device provides the user with a realistic virtual reality space via a VR headset.

[1133] 5. Emotion Recognition and Analysis

[1134] The user asks the samurai a question, and the device uses an emotion engine to analyze the tone of voice and facial expressions to obtain emotional data.

[1135] 6. Interaction and real-time response

[1136] The server generates the samurai's response in real time based on the user's emotional data and provides it to the user via the terminal.

[1137] Example: If a user asks the samurai with a smile, "How can I become stronger?", the samurai will reply, "Your enthusiasm is already proof of your strength."

[1138] In this way, users can not only experience a past era realistically and converse with historical figures, but also enjoy a more emotionally responsive and realistic experience by adjusting their responses and behavior according to the user's emotions. The present invention realizes a system that allows users to easily experience visiting a past era while providing an emotionally rich experience.

[1139] The processing flow will be explained below.

[1140] Step 1:

[1141] The user launches an application.

[1142] Action: The device confirms that the application is running and displays the login screen.

[1143] Step 2:

[1144] The user enters their login information and logs in.

[1145] Operation: The device sends the user's login information to the server and performs the authentication process.

[1146] Step 3:

[1147] The user selects a particular era or event.

[1148] How it works: The device receives the user's selection through the UI and sends a request to the server.

[1149] Step 4:

[1150] The server receives and parses the request.

[1151] How it works: The server analyzes the user's request and identifies the required data.

[1152] Step 5:

[1153] The server activates generative artificial intelligence and generates a virtual reality space.

[1154] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[1155] Step 6:

[1156] The server formats the virtual reality space and converts it into a VR-compatible data format.

[1157] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[1158] Step 7:

[1159] The server generates data on historical figures.

[1160] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[1161] Step 8:

[1162] The server generates character dialogue scripts using generative artificial intelligence.

[1163] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[1164] Step 9:

[1165] The server sends the VR data to the device.

[1166] Operation: The server sends the generated VR space data to the device.

[1167] Step 10:

[1168] The device transfers the data to the VR headset.

[1169] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[1170] Step 11:

[1171] The user interacts with the VR headset and controllers.

[1172] How it works: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with the characters.

[1173] Step 12:

[1174] The device analyzes the user's voice and facial expressions using an emotion engine.

[1175] How it works: The device analyzes emotional data from the user's tone of voice and facial expressions and sends that data to the server.

[1176] Step 13:

[1177] The server analyzes the user's behavioral and emotional data.

[1178] How it works: The server comprehensively analyzes the user's behavior and emotional data to determine the appropriate response of the historical figure.

[1179] Step 14:

[1180] The server sends the generated response data to the terminal.

[1181] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[1182] Step 15:

[1183] The terminal displays the response data to the user.

[1184] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[1185] Example 2

[1186] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1187] In conventional virtual reality systems, when users experience past eras, dialogue and interactions with historical figures cannot be adjusted in real time, resulting in a lack of realism and immersion. Furthermore, there was no system that could generate responses based on the user's emotional state, making it difficult to provide an experience that is more in tune with each individual user's emotions. Furthermore, the technical challenge of analyzing emotional data such as the user's voice and facial expressions and generating corresponding responses in real time remained unresolved.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This allows the user to realistically experience a past era, adjust dialogue and interaction with historical figures in real time, and generate responses according to the user's emotional state.

[1189] The "user request receiving means" is a means for the user to select a specific era or event and transmit that information to the system.

[1190] "Generative AI" is an AI technology used to recreate past eras, and is an AI capable of generating virtual reality spaces based on requests.

[1191] A "virtual reality space generation means" is a means for constructing a virtual reality space that recreates a specific era or event using generative artificial intelligence.

[1192] A "means for simulating historical figures" is a means for simulating the behavior and conversation of historical figures in a virtual reality space, providing users with realistic interactions.

[1193] The "virtual reality space providing means" refers to a device or method that provides the generated virtual reality space to the user and allows the user to interact within that space.

[1194] The "emotion engine" is an engine that analyzes the user's voice, facial expressions, body movements, etc., and recognizes their emotional state.

[1195] The "emotion data acquisition means" is a means for acquiring the user's emotion data analyzed using the emotion engine.

[1196] The "real-time response generation means" is a means for generating responses of characters in a virtual reality space in real time based on the user's behavior and emotional data.

[1197] The present invention relates to a system that allows users to experience past eras in a realistic way and enjoy direct communication with historical figures. Furthermore, the system provides a more natural and immersive experience by incorporating an emotion engine that recognizes the user's emotions. This system is implemented as follows.

[1198] System Overview

[1199] This system first receives a request for a specific era or event from the user. The user operates the terminal, selects a specific era or event from the past, and sends the request to the server. For example, a request might be, "I want to talk to a samurai in Japan during the Warring States period."

[1200] The server then uses generative artificial intelligence (e.g., GPT-4) to generate a virtual reality space based on the request. Specifically, it uses 3D rendering technology and generative AI models to create the scenery, buildings, and backgrounds of the requested era. For example, it generates a Japanese landscape, castle, or feudal lord's mansion from the Warring States period.

[1201] The server then generates behavior and conversation scripts for historical figures (such as samurai, feudal lords, and farmers) who will be active in the virtual reality space. This uses natural language processing technology to simulate realistic conversations. For example, it generates an object called a "samurai" and sets its behavioral patterns. The behavioral patterns include actions such as "patrolling the castle in the morning" and "talking when the user approaches."

[1202] The generated virtual reality space is provided to the user via a terminal. The user can move freely within the virtual space using a VR headset and controllers and interact with historical figures. For example, the user wears a VR headset and uses the controllers in their hands to walk and touch within the virtual space. The terminal renders the video and audio in real time.

[1203] The system also includes an emotion engine. The device uses the emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data. For example, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine then analyzes. The analyzed emotional data includes emotions such as joy, surprise, and sadness.

[1204] Finally, the server generates real-time responses from historical figures in the virtual reality space based on the user's behavioral and emotional data. For example, if the user shows a surprised expression, the samurai might respond, "Did I surprise you?" In this way, users can realistically experience past eras and feel a sense of presence when interacting with historical figures.

[1205] As a concrete example, consider the case where a user sends a request from a device saying, "I want to interact with a samurai in Sengoku-period Japan." In this case, the server generates the scenery, castles, and feudal lord mansions of Sengoku-period Japan, and sets up the actions and conversations of samurai, peasants, and feudal lord characters in the virtual reality space. The device then provides the user with a realistic virtual reality space via a VR headset, and when the user asks the samurai a question, the device obtains emotional data from the user's tone of voice and facial expressions, and generates the samurai's response in real time based on that data.

[1206] As a result, users can not only visit past eras, but also enjoy a more emotionally responsive and realistic experience, as responses and behaviors are adjusted according to the user's emotions.

[1207] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1208] Step 1:

[1209] Receiving a user request

[1210] The user selects a specific time period or event in the past and sends a request from the device to the server. The input is the time period or event selected by the user (e.g., "Japan during the Warring States period"). This input data is sent to the server via the device. Specifically, when the user makes a selection using the touchscreen and presses the confirmation button, the device forwards this to the server as an HTTP request. The output is data that the server has received and analyzed the request content.

[1211] Step 2:

[1212] Virtual reality space generation

[1213] The server analyzes the request and generates a virtual reality space using generative artificial intelligence (e.g., GPT-4). The input is information about a specific era or event received from the user. Based on this information, the server generates data for 3D rendering of landscapes, buildings, backgrounds, etc. Specifically, after analyzing the request, the server retrieves a dataset corresponding to "Japan during the Warring States period" and generates virtual castles and landscapes using a generative AI model. The output is data for the generated virtual reality space.

[1214] Step 3:

[1215] Simulation of historical figures

[1216] The server generates the behavior and conversation scripts of historical figures active within the virtual reality space. The inputs include data on the generated virtual reality space and information about the historical figures. Based on this, the server uses natural language processing technology to simulate realistic conversations. In terms of specific operations, the server generates an object called a "samurai" and sets its behavior patterns (e.g., "patrol the castle in the morning" or "talk to the user when they approach"). The output is the behavior patterns and conversation scripts of the generated historical figures.

[1217] Step 4:

[1218] Providing virtual reality space

[1219] The device provides the generated virtual reality space to the user. The input is virtual reality space data sent from the server. The device transfers this data to a VR headset and controller, allowing the user to move freely within the virtual space and interact with historical figures. Specifically, the user wears a VR headset and uses the controller in their hand to walk and touch within the virtual space. The output is the virtual reality space that the user actually experiences.

[1220] Step 5:

[1221] Emotion Recognition and Analysis

[1222] The device analyzes the user's voice, facial expressions, and body movements using an emotion engine to obtain emotion data. The input data includes the user's voice, facial expressions, and body movements. Specifically, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine analyzes (e.g., if the user smiles, it is recognized as "joy"). The output is the analyzed emotion data.

[1223] Step 6:

[1224] Generate real-time responses

[1225] The server generates a real-time response from the historical figure based on the user's behavior and emotion data. The input is the user's behavior data and emotion data. Specifically, the server receives the data sent from the emotion engine and adjusts the content of the historical figure's remarks (for example, if the user shows a surprised expression, the samurai will respond, "Did I surprise you?"). The output is the historical figure's real-time response.

[1226] This allows users to not only experience past eras in a realistic way and interact with historical figures, but also enjoy a more emotionally immersive experience as their responses and behavior are adjusted according to the user's emotions.

[1227] (Application example 2)

[1228] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] Conventional virtual reality systems lack emotional responses and interactions when users experience past times, making the experience less immersive. Additionally, factories and other historical sites tend to rely solely on modern technology and data to provide explanations, lacking in consideration of historical context and user emotions.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1231] In this invention, the server includes means for receiving requests for specific eras or events from users, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures within the virtual reality space, thereby enabling users to experience a past era with a sense of realism that includes emotions.

[1232] The "means for receiving a request for a specific era or event from a user" is an interface or device that allows a user to select and request a specific era or event that the user wishes to experience.

[1233] "Generative AI" is an AI technology that automatically generates past eras and scenarios based on user requests.

[1234] "Means for generating virtual reality spaces" refers to devices or software that use generative artificial intelligence to create three-dimensional virtual spaces that faithfully recreate past eras and events.

[1235] "Means for simulating the behavior and conversation of historical figures within a virtual reality space" refers to devices or software for simulating in real time the movements and conversation of historical figures appearing within a virtual reality space.

[1236] "Means for providing a virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with" means means for providing the generated virtual reality space to a user and transmitting data to a headset or other device for the user to experience and interact within the space.

[1237] An "emotion engine" is software that analyzes and recognizes emotions from a user's voice, facial expressions, actions, etc.

[1238] The "means for generating responses of historical figures in real time based on emotion data" is a technology for generating responses and behaviors of historical figures in real time based on the emotion data of a user.

[1239] This invention relates to a virtual reality system that allows users to experience past eras realistically and interact with historical figures. By combining it with an emotion engine, it provides real-time responses and behaviors according to the user's emotions, realizing a more natural and immersive experience.

[1240] System Program Overview

[1241] The server receives requests from users for specific eras or events. For example, if a user requests a "factory tour from the early 20th century," the server analyzes the request and uses generative artificial intelligence to generate a virtual reality space. This virtual reality space realistically reproduces the factory's scenery, the operation of machines, and the behavior of workers.

[1242] The server then simulates the behavior and conversation of historical figures (in this case, factory workers and engineers) within the virtual reality space, enabling realistic interactions within the virtual reality space experienced by the user.

[1243] The generated virtual reality space is provided to the user via a VR headset and controller, allowing the user to experience a past era.

[1244] Hardware and Software Details

[1245] VR headset: A device that provides users with a virtual reality space.

[1246] Robot guide: A robot for interacting with users.

[1247] Emotion recognition sensor: A sensor that acquires emotions from the user's voice, facial expressions, and movements.

[1248] OpenAI GPT-3: As a generative artificial intelligence, it is used to analyze user requests and generate real-time responses based on virtual reality spatial data and emotions.

[1249] Emotion Recognition Library: A library for analyzing user emotions and obtaining emotional data.

[1250] VR Renderer: Rendering software for generating virtual reality spaces.

[1251] Robot Controller software: Software for controlling the operation of the robot guide.

[1252] Specific examples

[1253] For example, if a user requests "I want to experience a factory tour from the early 20th century," the server generates a virtual reality space using the following prompts:

[1254] "A user requests a tour of an early 20th century factory. What kind of scene would you generate?"

[1255] In the generated virtual reality space, old machinery and work scenes in the factory are realistically reproduced. If the user is excited, the emotion recognition sensor will recognize that emotion and the emotion engine will analyze the data. For example, if the user excitedly asks, "How does this machine work?", the robot guide can respond, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[1256] In this way, a system is realized that allows users to experience a past era with a sense of realism while engaging in emotional interaction.

[1257] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1258] Step 1:

[1259] The user inputs a request to experience a specific era or event. Specifically, the user sends a request via their terminal saying, "I would like to experience a factory tour from the early 20th century." This input data is sent to the server, which receives and analyzes the request.

[1260] Step 2:

[1261] Based on the received request, the server generates prompts to generate a virtual reality space using a generative AI model (OpenAI GPT-3). The input is the user's request, and the output is virtual data that defines the specific scenario and components of the virtual reality space. For example, it generates a prompt such as, "The user has requested a factory tour from the early 20th century. What kind of scene would you like to generate?"

[1262] Step 3:

[1263] The server inputs prompts into the generative AI model to generate details of the virtual reality space. The input is the prompt sentence, and the output is detailed virtual reality space data (e.g., factory scenery, machine operation, worker behavior, etc.). The server passes this output to the VR Renderer, which renders the virtual reality space.

[1264] Step 4:

[1265] The terminal receives the rendered virtual reality space data and provides it to the user via a VR headset. The user wears the VR headset and virtually experiences an early 20th century factory. The input is the rendered virtual reality space data, and the output is a virtual environment experienced by the user visually and aurally.

[1266] Step 5:

[1267] The user interacts with the virtual reality space. For example, the user asks, "How does this machine work?" The device acquires the user's voice data and sends it to the server. The input is the user's voice data, and the output is the transmission of the voice data to the server.

[1268] Step 6:

[1269] The server uses an emotion recognition sensor to recognize emotions from the user's voice data. The input is voice data, and the output is emotion data (e.g., excitement, curiosity). The server analyzes this emotion data with an emotion engine.

[1270] Step 7:

[1271] The server uses a generative AI model to generate real-time responses based on the user's emotions, based on the emotion data acquired by the emotion engine. The input is the emotion data and the user's voice question, and the output is an appropriate response (e.g., "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time").

[1272] Step 8:

[1273] The terminal receives response data from the server and responds to the user in real time via the robot guide. The input is the response data, and the output is a voice response to the user. Specifically, the robot guide says, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[1274] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1275] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1276] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1277] [Fourth embodiment]

[1278] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1279] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1280] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1281] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1282] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1283] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1284] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1285] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1286] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1287] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1288] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1289] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1290] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1291] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures. The system of the present invention is implemented in the following manner.

[1292] System Overview

[1293] The system receives requests from users for specific eras or events, uses generative artificial intelligence to generate a virtual reality space based on the request, simulates historical figures within the virtual reality space, and provides the virtual reality space to a device for user interaction. It then generates real-time responses to the user's actual actions within the virtual reality space.

[1294] Program processing

[1295] 1. Receiving a user request

[1296] The terminal receives a request from the user selecting a past era or event and transmits it to the server.

[1297] Example: A user requests Japan during the Sengoku period.

[1298] 2. Creation of virtual reality space

[1299] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[1300] Example: The server generates backgrounds, scenery, and buildings for Japan during the Sengoku period.

[1301] 3. Simulation of historical figures

[1302] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[1303] Example: The server specifies the samurai's behavior patterns and conversation content.

[1304] 4. Provision of virtual reality space

[1305] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[1306] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[1307] 5. Generating real-time responses

[1308] The server generates responses from historical figures in real time based on the user's actions.

[1309] Example: When a user speaks to a samurai, the server uses generative artificial intelligence to generate the samurai's response and sends it to the device.

[1310] Specific examples

[1311] Example 1: Interacting with a samurai in Sengoku period Japan

[1312] 1. User Request

[1313] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[1314] 2. Creation of virtual reality space

[1315] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[1316] 3. Simulation of historical figures

[1317] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[1318] 4. Provision of virtual reality space

[1319] The device provides the user with a realistic virtual reality space via a VR headset.

[1320] 5. Interaction and real-time response

[1321] When the user talks to the samurai, the server generates the samurai's response in real time and sends it to the terminal. The samurai will respond with something like, "Can you tell me about your experience in today's battle?"

[1322] In this way, users can realistically experience past eras and enjoy conversations with historical figures. The present invention provides a system that allows users to easily experience visiting past eras.

[1323] The processing flow will be explained below.

[1324] Step 1:

[1325] The user launches an application.

[1326] Action: The device confirms that the application is running and displays the login screen.

[1327] Step 2:

[1328] The user enters their login information and logs in.

[1329] Operation: The device sends the user's login information to the server and performs the authentication process.

[1330] Step 3:

[1331] The user selects a particular era or event.

[1332] How it works: The device receives the user's selection through the UI and sends a request to the server.

[1333] Step 4:

[1334] The server receives and parses the request.

[1335] How it works: The server analyzes the user's request and identifies the required data.

[1336] Step 5:

[1337] The server activates generative artificial intelligence and generates a virtual reality space.

[1338] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[1339] Step 6:

[1340] The server formats the virtual reality space and converts it into a VR-compatible data format.

[1341] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[1342] Step 7:

[1343] The server generates data on historical figures.

[1344] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[1345] Step 8:

[1346] The server generates character dialogue scripts using generative artificial intelligence.

[1347] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[1348] Step 9:

[1349] The server sends the VR data to the device.

[1350] Operation: The server sends the generated VR space data to the device.

[1351] Step 10:

[1352] The device transfers the data to the VR headset.

[1353] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[1354] Step 11:

[1355] The user interacts with the VR headset and controllers.

[1356] Actions: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with characters.

[1357] Step 12:

[1358] The device detects the user's actions and sends them to the server.

[1359] How it works: The device detects the user's actions (e.g. talking to the samurai) and sends that information to the server.

[1360] Step 13:

[1361] The server analyzes the user's behavior and generates an appropriate response.

[1362] How it works: The server analyzes the user's behavior and uses generative artificial intelligence to generate real-time responses from historical figures.

[1363] Step 14:

[1364] The server sends the generated response data to the terminal.

[1365] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[1366] Step 15:

[1367] The terminal displays the response data to the user.

[1368] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[1369] Example 1

[1370] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1371] Conventional virtual reality systems, when users experience past eras or historical events, have problems with insufficient real-time interaction and response, and limited dialogue and behavior with historical characters. Furthermore, there is a lack of technology to generate highly accurate virtual reality spaces based on user requests. To address these issues, a system that can achieve real-time response and precise simulation by utilizing generative artificial intelligence is needed.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1373] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, means for simulating the behavior and conversation of historical figures in the virtual reality space, means for providing the virtual reality space to the user and transmitting the virtual reality space to a device with which the user interacts, means for monitoring interaction data from the user and generating responses in real time, means for generating 3D models of historical backgrounds, scenery, and buildings using generative artificial intelligence, and means for transmitting data of the generated virtual reality space to the user's device, thereby enabling the user to realistically experience past eras and enjoy conversations with historical figures.

[1374] "User" refers to any individual or entity that uses the System.

[1375] A "request" is a request in the form of a user inputting into a terminal their intention to experience a specific era or event.

[1376] "Terminal" refers to the device through which a user accesses the system, inputs requests, and experiences the virtual reality space, including, but not limited to, a smartphone, PC, or tablet.

[1377] "Server" refers to a central computer or cloud service that receives requests from users, uses generative artificial intelligence to simulate virtual reality spaces and historical figures, and sends the results to terminals.

[1378] "Generative AI" refers to AI technology that generates the actions and conversations of past eras and historical figures based on user requests. Specifically, it includes machine learning models and natural language processing models.

[1379] "Virtual reality space" refers to a 3D computer graphics environment created using generative artificial intelligence that recreates a specific era or event.

[1380] "Historical figures" refer to characters who actually existed in the past or who are spoken of as legends, and within this system their actions and conversations are simulated by generative artificial intelligence.

[1381] "Behavior" refers to the actions and movements of historical figures within the virtual reality space.

[1382] "Conversation" refers to a dialogue between a historical figure and the user or another character within a virtual reality space.

[1383] "Simulation" refers to the process of realistically recreating the behavior and conversations of historical figures in a virtual reality space using generative artificial intelligence.

[1384] "Interaction" refers to the operations and actions performed by the user within the virtual reality space, including the system's response to them.

[1385] "Real-time responses" refer to the responses and actions of historical figures that are generated instantly in response to user interactions.

[1386] "3D model" refers to a three-dimensional computer graphics object created by generative artificial intelligence, and specifically includes backgrounds, landscapes, buildings, etc.

[1387] The system of the present invention allows users to realistically experience past eras and enjoy direct communication with historical figures. Specific embodiments for carrying out the present invention will now be described in detail.

[1388] First, a user inputs a request to experience a specific era or event into a terminal. The terminal can be a smartphone, PC, tablet, or other device, and sends the user's input as an HTTP request to a server. The server then receives this request and generates a virtual reality space using generative artificial intelligence. Generative artificial intelligence uses machine learning models and natural language processing models, such as OpenAI's GPT model.

[1389] Based on user requests, the server retrieves information on historical background, landscapes, buildings, and other aspects, and performs advanced 3D modeling. Game engines such as Unity and Unreal Engine are used for this. The server also uses generative artificial intelligence to simulate the behavior and conversation of historical figures in the virtual reality space. The simulation includes characters such as samurai, feudal lords, and farmers.

[1390] The generated virtual reality space data is sent from the server to the user's device. The user wears a VR headset (e.g., Oculus Rift or HTC Vive) and experiences the virtual reality space. The data displayed on the VR headset includes Japanese landscapes, castles, and feudal lord mansions from the Warring States period, and the user can interact with them using the VR controller.

[1391] When a user performs a specific action in the virtual reality space, that data is sent to the server in real time, and the server uses generative artificial intelligence to generate an appropriate response. For example, when a user speaks to a samurai, the server generates the samurai's response and sends it to the device. If the user asks the samurai, "Will you tell me about your experience in today's battle?", the samurai can instantly generate and display a response such as, "I'll tell you about my experience in today's battle."

[1392] Specific examples

[1393] Example 1: Interacting with a samurai in Sengoku period Japan

[1394] The user inputs a request into the device saying, "I want to interact with a samurai in Sengoku-period Japan." This request is sent to the server as an HTTP request. The server uses generative artificial intelligence to generate Sengoku-period scenery, castles, and feudal lord mansions. The generated virtual reality space data is sent to the device, and the user experiences the virtual reality space using a VR headset. When the user asks the samurai, "Will you tell me about your experience in today's battle?", the server uses generative artificial intelligence to generate a response from the samurai, and the response "I'll tell you about my experience in today's battle" is displayed.

[1395] Prompt Sentence Examples

[1396] "I want to talk to a samurai from the Sengoku period. I want to learn his techniques."

[1397] "I would like to see a feudal lord's mansion. I would like to know what kind of decorations and furniture were there."

[1398] "I want to experience a day in the life of a farmer. I want to see what kind of work they do."

[1399] In this way, users can experience a past era in a realistic way and enjoy conversations with historical figures. This invention allows users to easily experience visiting a past era.

[1400] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1401] Step 1:

[1402] Receiving a user request

[1403] Input: The user types a request into the terminal to select a particular era or event.

[1404] Specific operation: The user inputs a request saying, "I want to interact with a samurai in Japan during the Warring States period."

[1405] Output: The terminal sends the user's request to the server as an HTTP request.

[1406] Step 2:

[1407] Processing virtual reality space generation requests

[1408] Input: The user request sent from the terminal.

[1409] Specific operation: The server receives the request and constructs a prompt to generate the virtual reality space using generative artificial intelligence (e.g., OpenAI's GPT model).

[1410] Output: Constructing prompts for generative artificial intelligence.

[1411] Step 3:

[1412] Virtual reality space generation

[1413] Input: A prompt for generative artificial intelligence.

[1414] How it works: The server generates 3D models of Japanese landscapes, castles, feudal lord mansions, etc. from the Warring States period based on prompts, using game engines such as Unity or Unreal Engine.

[1415] Output: 3D model data of the generated virtual reality space.

[1416] Step 4:

[1417] Simulation of historical figures

[1418] Input: 3D model data of the virtual reality space and additional prompt data.

[1419] Specific operation: The server uses generative artificial intelligence to generate the behavior and conversation scripts of historical figures in the virtual reality space. It generates the behavior patterns and conversation content of characters such as samurai, feudal lords, and farmers.

[1420] Output: Simulation data of the generated historical figures.

[1421] Step 5:

[1422] Integration and provision of virtual reality spaces

[1423] Input: 3D model data of virtual reality space and simulation data of historical figures.

[1424] How it works: The server aggregates the virtual reality data and sends it to the user's device, which then provides the virtual reality space to the user via a VR headset.

[1425] Output: The virtual reality space is displayed on the user's device.

[1426] Step 6:

[1427] Monitor interactions and generate real-time responses

[1428] Input: User behavior data (e.g., voice, movement).

[1429] How it works: The device sends user behavior data to the server, which then uses generative artificial intelligence to generate real-time responses based on the user's behavior. For example, when the user speaks to the samurai, the voice data is sent to the server, which then generates an appropriate response from the samurai.

[1430] Output: The generated real-time response data is sent to the terminal and displayed as an appropriate character's response in the virtual reality space.

[1431] As a specific example, if a user requests, "I want to interact with a samurai in Japan during the Warring States period," the user will go through the above steps to be immersed in a virtual reality space of the Warring States period and be able to enjoy real-time conversations with historical figures. This system makes it easy to experience visiting a past era.

[1432] (Application example 1)

[1433] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1434] Conventional virtual reality experience systems lacked the technology to enable real-time interaction with past eras or historical figures. In particular, it was difficult to generate intuitive and natural responses to user actions and interactions, limiting their ability to provide a realistic experience. Furthermore, there was no easy way to achieve such an experience using mobile devices such as smartphones. This resulted in a lack of interactivity in historical learning and entertainment.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1436] In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era based on the request using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This enables a means for experiencing the virtual reality space using a smartphone and providing an interface for interacting with characters, and means for analyzing the conversation between the historical figure and the user using a natural language processing model and generating an appropriate response based on the content of the conversation.

[1437] definition statement

[1438] The "means for receiving a request from a user for a specific era or event" refers to a means for receiving the request within the system through an interface for a user to input a request for a past era or a specific event.

[1439] "Generative AI" is AI that has the ability to generate new data based on large amounts of data according to specified parameters.

[1440] A "means for generating a virtual reality space" is a means for recreating a past era on a computer based on a received request, and constructing a virtual environment that can be used visually and experientially by the user.

[1441] "Means for simulating the behavior and conversation of historical figures" refers to a means of imitating the behavior patterns and speaking styles of people who lived in past eras using generative artificial intelligence, enabling interaction with users.

[1442] "Devices for user interaction" are devices or interfaces that allow users to access a virtual reality space and perform operations and interactions within that space.

[1443] "Means for generating responses in real time" refers to a means for instantly generating appropriate responses to the user's actions and statements, and replying as a historical figure within the virtual reality space.

[1444] "Means for providing an interface for experiencing a virtual reality space using a smartphone and interacting with characters" refers to a means for displaying a virtual reality space using a smartphone as a medium and for users to interact with characters within that space.

[1445] A "natural language processing model" is an artificial intelligence-based model for analyzing human language and understanding and generating its meaning and context.

[1446] "Means for generating appropriate responses" refers to techniques for providing natural and relevant replies based on the content of the dialogue with the user.

[1447] System configuration

[1448] This invention is composed of a user, a server, and a terminal. The user operates the system using a smartphone to request a specific era or event.

[1449] Program processing

[1450] 1. Receiving a user request

[1451] Users use a smartphone interface to request a specific time period or event from the past, which is then sent over the internet to a server.

[1452] 2. Creation of virtual reality space

[1453] The server uses OpenAI's generative AI model (e.g., GPT-3) to generate a virtual reality space based on the user's request, faithfully recreating historical and cultural elements.

[1454] 3. Simulation of historical figures

[1455] The server also uses generative AI models to simulate the behavior and conversation of historical figures, referencing a database of past events to ensure the simulated characters act and speak in a way appropriate to the time period.

[1456] 4. Provision of virtual reality space

[1457] The virtual reality space and simulated historical figures generated by the server are sent to the user's smartphone, where the user can access the virtual reality space and experience it using devices such as a VR headset and controller.

[1458] 5. Generating real-time responses

[1459] The server generates real-time responses to user interactions. When a user speaks to a historical figure, the content is analyzed using a natural language processing model, and an appropriate response is returned.

[1460] Hardware and Software

[1461] Hardware: Smartphone (iOS or Android compatible), VR headset

[1462] software:

[1463] OpenAI GPT-3 API: Used as a generative AI model to generate historical context and character behavior.

[1464] Unity3D: A platform and engine for building VR environments.

[1465] REST API: An interface for data communication between the server and the app.

[1466] Specific examples

[1467] If the user requests to interact with a warlord in Sengoku-era Japan, the following prompt sentence is used:

[1468] I would like to talk to warlords in Japan during the Sengoku period and learn about their daily lives and battles.

[1469] Based on this request, the system recreates the background, scenery, castles, and mansions of the Sengoku period, and generates the behavioral patterns and conversations of the warlords. When the user enters the virtual reality space using a VR headset and talks to the warlord, the warlord will respond in real time with a response such as, "Today was a tough battle. I fought and risked my life in it. It's time to show true courage."

[1470] This invention provides a system that allows users to easily experience visiting past eras in a realistic way using a smartphone and a generative AI model.

[1471] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1472] Program processing steps

[1473] Step 1:

[1474] Receiving a user request

[1475] explanation

[1476] Users launch the smartphone application and input the past era or event they wish to experience. This input is done by sending a request to the application screen, such as "I want to experience Japan during the Warring States period." The request is then sent to the server through the application interface.

[1477] Input and Output

[1478] Input: A request typed by a user through a smartphone interface (e.g., "I want to experience Japan during the Warring States period").

[1479] Output: The request data sent to the server.

[1480] operation

[1481] Specific operation: The user enters the desired era and event in the input form and presses the "Submit" button. The application sends the request data to the server.

[1482] Step 2:

[1483] Virtual reality space generation

[1484] explanation

[1485] The server processes the received request and sends a prompt to OpenAI's generative AI model (GPT-3). The prompt conveys details of the requested era and event to the generative AI model. The generative AI model generates background data for the virtual reality space based on the prompt.

[1486] Input and Output

[1487] Input: The request data received by the server and a prompt (e.g., "Recreate the setting and key events of Japan during the Warring States period.").

[1488] Output: Background data of the generated virtual reality space.

[1489] operation

[1490] How it works: The server takes in the request data, generates a prompt based on it, and sends it to the OpenAI API. The generative AI model generates background data for the virtual reality space in response to the prompt and returns it to the server.

[1491] Step 3:

[1492] Simulation of historical figures

[1493] explanation

[1494] The server uses a generative AI model to generate behavior and conversation data for historical figures, which involves referencing a historical database to set behavior patterns and conversation content appropriate for that era.

[1495] Input and Output

[1496] Input: Background data of the virtual reality space generated by the server and a prompt (e.g., "Generate the behavior patterns and speech patterns of a warlord from the Sengoku period.").

[1497] Output: Behavioral and conversational data of the generated historical figures.

[1498] operation

[1499] Specific operation: The server generates a more detailed prompt sentence and sends it to the generative AI model. The generative AI model generates behavior and conversation data of the historical figure based on the prompt and replies to the server.

[1500] Step 4:

[1501] Providing virtual reality space

[1502] explanation

[1503] The server sends the generated virtual reality space and simulation data to the user's smartphone, where the user can access the virtual reality space using an application.

[1504] Input and Output

[1505] Input: Generated virtual reality space and simulation data of historical figures.

[1506] Output: VR data sent to the user's smartphone.

[1507] operation

[1508] Specific operation: The server packages the generated virtual reality space data and sends it to the user's smartphone using a REST API. The user then accesses the virtual reality space through the smartphone screen and begins the experience.

[1509] Step 5:

[1510] Generate real-time responses

[1511] explanation

[1512] When a user interacts with a historical figure in the virtual reality space, the content of the interaction is transmitted to the server, which then analyzes the interaction using a natural language processing model and generates an appropriate response. The generated response is then transmitted in real time to the user's smartphone and displayed in the virtual reality space.

[1513] Input and Output

[1514] Input: User interaction (e.g., "How was your battle today?").

[1515] Output: The generated response of the historical figure (e.g., "Today was a tough battle. I risked my life in it. It's time to show true courage.").

[1516] operation

[1517] How it works: When a user speaks through the microphone in their VR headset, the voice data is sent to the server. The server uses a natural language processing model to convert the voice data into text, analyzes the text, and sends it to the generative AI model. The generative AI model generates a response text and sends it back to the server. The server then sends this response to the user's smartphone and displays it in the virtual reality space.

[1518] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1519] The present invention relates to a system that enables a user to realistically experience a past era and enjoy direct communication with historical figures, and further relates to a system that provides a more natural and realistic experience by combining it with an emotion engine that recognizes the user's emotions. The system of the present invention is implemented in the following form.

[1520] System Overview

[1521] This system receives requests from users for specific eras or events, and uses generative artificial intelligence to generate a virtual reality space based on the request. It then simulates historical figures within the virtual reality space and provides the virtual reality space to a device for user interaction. It also uses an emotion engine to recognize the user's emotions, and adjusts the responses and behavior of the historical figures in the virtual reality space in real time based on the recognition results, providing a more emotionally relevant experience for the user.

[1522] Program processing

[1523] 1. Receiving a user request

[1524] The user selects a past era or event and sends a request from the terminal to the server.

[1525] Example: A user requests Japan during the Sengoku period.

[1526] 2. Creation of virtual reality space

[1527] The server uses generative artificial intelligence to generate a virtual reality space based on the request.

[1528] Example: The server generates the scenery, buildings and backgrounds of Japan during the Sengoku period.

[1529] 3. Simulation of historical figures

[1530] The server generates behavior and dialogue scripts for historical figures (samurai, feudal lords, farmers, etc.).

[1531] Example: The server sets the samurai's behavior patterns and conversation content.

[1532] 4. Provision of virtual reality space

[1533] The device provides the user with a virtual reality space, which the user interacts with using a VR headset and controller.

[1534] Example: A user puts on a VR headset and experiences Japan during the Sengoku period.

[1535] 5. Emotion Recognition and Analysis

[1536] The device uses an emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data.

[1537] Example: The device recognizes emotions such as joy, surprise, and sadness from the user's tone of voice and facial expressions.

[1538] 6. Generating real-time responses

[1539] The server generates real-time responses of historical figures based on the user's behavioral and emotional data.

[1540] Example: The user asks the samurai a question, and the samurai responds kindly according to the user's emotions.

[1541] Specific examples

[1542] Example 1: Interacting with a samurai in Sengoku period Japan

[1543] 1. User Request

[1544] The user sends a request from their device saying, "I want to interact with a samurai in Japan during the Warring States period."

[1545] 2. Creation of virtual reality space

[1546] The server uses generative artificial intelligence to generate Japanese landscapes, castles, feudal lord mansions, and other scenes from the Warring States period.

[1547] 3. Simulation of historical figures

[1548] The server generates the actions and conversations of characters such as samurai, farmers, and feudal lords.

[1549] 4. Provision of virtual reality space

[1550] The device provides the user with a realistic virtual reality space via a VR headset.

[1551] 5. Emotion Recognition and Analysis

[1552] The user asks the samurai a question, and the device uses an emotion engine to analyze the tone of voice and facial expressions to obtain emotional data.

[1553] 6. Interaction and real-time response

[1554] The server generates the samurai's response in real time based on the user's emotional data and provides it to the user via the terminal.

[1555] Example: If a user asks the samurai with a smile, "How can I become stronger?", the samurai will reply, "Your enthusiasm is already proof of your strength."

[1556] In this way, users can not only experience a past era realistically and converse with historical figures, but also enjoy a more emotionally responsive and realistic experience by adjusting their responses and behavior according to the user's emotions. The present invention realizes a system that allows users to easily experience visiting a past era while providing an emotionally rich experience.

[1557] The processing flow will be explained below.

[1558] Step 1:

[1559] The user launches an application.

[1560] Action: The device confirms that the application is running and displays the login screen.

[1561] Step 2:

[1562] The user enters their login information and logs in.

[1563] Operation: The device sends the user's login information to the server and performs the authentication process.

[1564] Step 3:

[1565] The user selects a particular era or event.

[1566] How it works: The device receives the user's selection through the UI and sends a request to the server.

[1567] Step 4:

[1568] The server receives and parses the request.

[1569] How it works: The server analyzes the user's request and identifies the required data.

[1570] Step 5:

[1571] The server activates generative artificial intelligence and generates a virtual reality space.

[1572] How it works: The server inputs data into a generative artificial intelligence to generate the landscapes, buildings, and backgrounds of Japan during the Sengoku period.

[1573] Step 6:

[1574] The server formats the virtual reality space and converts it into a VR-compatible data format.

[1575] How it works: The server formats the generated data and converts it into a format that can be displayed on a VR headset.

[1576] Step 7:

[1577] The server generates data on historical figures.

[1578] How it works: The server generates characters of historical figures from the Sengoku period, such as samurai, feudal lords, and farmers, and sets their behavior patterns and behaviour.

[1579] Step 8:

[1580] The server generates character dialogue scripts using generative artificial intelligence.

[1581] How it works: The server generates a dialogue script for each character and sets appropriate response patterns.

[1582] Step 9:

[1583] The server sends the VR data to the device.

[1584] Operation: The server sends the generated VR space data to the device.

[1585] Step 10:

[1586] The device transfers the data to the VR headset.

[1587] How it works: The device transfers the received data to the VR headset, displaying the virtual reality space to the user.

[1588] Step 11:

[1589] The user interacts with the VR headset and controllers.

[1590] How it works: The user puts on a VR headset and uses the controllers to move around in the virtual reality space and interact with the characters.

[1591] Step 12:

[1592] The device analyzes the user's voice and facial expressions using an emotion engine.

[1593] How it works: The device analyzes emotional data from the user's tone of voice and facial expressions and sends that data to the server.

[1594] Step 13:

[1595] The server analyzes the user's behavioral and emotional data.

[1596] How it works: The server comprehensively analyzes the user's behavior and emotional data to determine the appropriate response of the historical figure.

[1597] Step 14:

[1598] The server sends the generated response data to the terminal.

[1599] Operation: The server generates response data and sends it to the terminal, providing the user with a real-time response.

[1600] Step 15:

[1601] The terminal displays the response data to the user.

[1602] Operation: The response data received by the device is reflected in the VR space, allowing the user to experience the response in real time.

[1603] Example 2

[1604] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1605] In conventional virtual reality systems, when users experience past eras, dialogue and interactions with historical figures cannot be adjusted in real time, resulting in a lack of realism and immersion. Furthermore, there was no system that could generate responses based on the user's emotional state, making it difficult to provide an experience that is more in tune with each individual user's emotions. Furthermore, the technical challenge of analyzing emotional data such as the user's voice and facial expressions and generating corresponding responses in real time remained unresolved.

[1606] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving a request for a specific era or event from a user, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures in the virtual reality space. This allows the user to realistically experience a past era, adjust dialogue and interaction with historical figures in real time, and generate responses according to the user's emotional state.

[1607] The "user request receiving means" is a means for the user to select a specific era or event and transmit that information to the system.

[1608] "Generative AI" is an AI technology used to recreate past eras, and is an AI capable of generating virtual reality spaces based on requests.

[1609] A "virtual reality space generation means" is a means for constructing a virtual reality space that recreates a specific era or event using generative artificial intelligence.

[1610] A "means for simulating historical figures" is a means for simulating the behavior and conversation of historical figures in a virtual reality space, providing users with realistic interactions.

[1611] The "virtual reality space providing means" refers to a device or method that provides the generated virtual reality space to the user and allows the user to interact within that space.

[1612] The "emotion engine" is an engine that analyzes the user's voice, facial expressions, body movements, etc., and recognizes their emotional state.

[1613] The "emotion data acquisition means" is a means for acquiring the user's emotion data analyzed using the emotion engine.

[1614] The "real-time response generation means" is a means for generating responses of characters in a virtual reality space in real time based on the user's behavior and emotional data.

[1615] The present invention relates to a system that allows users to experience past eras in a realistic way and enjoy direct communication with historical figures. Furthermore, the system provides a more natural and immersive experience by incorporating an emotion engine that recognizes the user's emotions. This system is implemented as follows.

[1616] System Overview

[1617] This system first receives a request for a specific era or event from the user. The user operates the terminal, selects a specific era or event from the past, and sends the request to the server. For example, a request might be, "I want to talk to a samurai in Japan during the Warring States period."

[1618] The server then uses generative artificial intelligence (e.g., GPT-4) to generate a virtual reality space based on the request. Specifically, it uses 3D rendering technology and generative AI models to create the scenery, buildings, and backgrounds of the requested era. For example, it generates a Japanese landscape, castle, or feudal lord's mansion from the Warring States period.

[1619] The server then generates behavior and conversation scripts for historical figures (such as samurai, feudal lords, and farmers) who will be active in the virtual reality space. This uses natural language processing technology to simulate realistic conversations. For example, it generates an object called a "samurai" and sets its behavioral patterns. The behavioral patterns include actions such as "patrolling the castle in the morning" and "talking when the user approaches."

[1620] The generated virtual reality space is provided to the user via a terminal. The user can move freely within the virtual space using a VR headset and controllers and interact with historical figures. For example, the user wears a VR headset and uses the controllers in their hands to walk and touch within the virtual space. The terminal renders the video and audio in real time.

[1621] The system also includes an emotion engine. The device uses the emotion engine to analyze the user's voice, facial expressions, and body movements to obtain emotional data. For example, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine then analyzes. The analyzed emotional data includes emotions such as joy, surprise, and sadness.

[1622] Finally, the server generates real-time responses from historical figures in the virtual reality space based on the user's behavioral and emotional data. For example, if the user shows a surprised expression, the samurai might respond, "Did I surprise you?" In this way, users can realistically experience past eras and feel a sense of presence when interacting with historical figures.

[1623] As a concrete example, consider the case where a user sends a request from a device saying, "I want to interact with a samurai in Sengoku-period Japan." In this case, the server generates the scenery, castles, and feudal lord mansions of Sengoku-period Japan, and sets up the actions and conversations of samurai, peasants, and feudal lord characters in the virtual reality space. The device then provides the user with a realistic virtual reality space via a VR headset, and when the user asks the samurai a question, the device obtains emotional data from the user's tone of voice and facial expressions, and generates the samurai's response in real time based on that data.

[1624] As a result, users can not only visit past eras, but also enjoy a more emotionally responsive and realistic experience, as responses and behaviors are adjusted according to the user's emotions.

[1625] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1626] Step 1:

[1627] Receiving a user request

[1628] The user selects a specific time period or event in the past and sends a request from the device to the server. The input is the time period or event selected by the user (e.g., "Japan during the Warring States period"). This input data is sent to the server via the device. Specifically, when the user makes a selection using the touchscreen and presses the confirmation button, the device forwards this to the server as an HTTP request. The output is data that the server has received and analyzed the request content.

[1629] Step 2:

[1630] Virtual reality space generation

[1631] The server analyzes the request and generates a virtual reality space using generative artificial intelligence (e.g., GPT-4). The input is information about a specific era or event received from the user. Based on this information, the server generates data for 3D rendering of landscapes, buildings, backgrounds, etc. Specifically, after analyzing the request, the server retrieves a dataset corresponding to "Japan during the Warring States period" and generates virtual castles and landscapes using a generative AI model. The output is data for the generated virtual reality space.

[1632] Step 3:

[1633] Simulation of historical figures

[1634] The server generates the behavior and conversation scripts of historical figures active within the virtual reality space. The inputs include data on the generated virtual reality space and information about the historical figures. Based on this, the server uses natural language processing technology to simulate realistic conversations. In terms of specific operations, the server generates an object called a "samurai" and sets its behavior patterns (e.g., "patrol the castle in the morning" or "talk to the user when they approach"). The output is the behavior patterns and conversation scripts of the generated historical figures.

[1635] Step 4:

[1636] Providing virtual reality space

[1637] The device provides the generated virtual reality space to the user. The input is virtual reality space data sent from the server. The device transfers this data to a VR headset and controller, allowing the user to move freely within the virtual space and interact with historical figures. Specifically, the user wears a VR headset and uses the controller in their hand to walk and touch within the virtual space. The output is the virtual reality space that the user actually experiences.

[1638] Step 5:

[1639] Emotion Recognition and Analysis

[1640] The device analyzes the user's voice, facial expressions, and body movements using an emotion engine to obtain emotion data. The input data includes the user's voice, facial expressions, and body movements. Specifically, the device captures the user's facial expressions and voice using the camera and microphone built into the VR headset, which the emotion engine analyzes (e.g., if the user smiles, it is recognized as "joy"). The output is the analyzed emotion data.

[1641] Step 6:

[1642] Generate real-time responses

[1643] The server generates a real-time response from the historical figure based on the user's behavior and emotion data. The input is the user's behavior data and emotion data. Specifically, the server receives the data sent from the emotion engine and adjusts the content of the historical figure's remarks (for example, if the user shows a surprised expression, the samurai will respond, "Did I surprise you?"). The output is the historical figure's real-time response.

[1644] This allows users to not only experience past eras in a realistic way and interact with historical figures, but also enjoy a more emotionally immersive experience as their responses and behavior are adjusted according to the user's emotions.

[1645] (Application example 2)

[1646] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] Conventional virtual reality systems lack emotional responses and interactions when users experience past times, making the experience less immersive. Additionally, factories and other historical sites tend to rely solely on modern technology and data to provide explanations, lacking in consideration of historical context and user emotions.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1649] In this invention, the server includes means for receiving requests for specific eras or events from users, means for generating a virtual reality space that recreates a past era using generative artificial intelligence, and means for simulating the behavior and conversation of historical figures within the virtual reality space, thereby enabling users to experience a past era with a sense of realism that includes emotions.

[1650] The "means for receiving a request for a specific era or event from a user" is an interface or device that allows a user to select and request a specific era or event that the user wishes to experience.

[1651] "Generative AI" is an AI technology that automatically generates past eras and scenarios based on user requests.

[1652] "Means for generating virtual reality spaces" refers to devices or software that use generative artificial intelligence to create three-dimensional virtual spaces that faithfully recreate past eras and events.

[1653] "Means for simulating the behavior and conversation of historical figures within a virtual reality space" refers to devices or software for simulating in real time the movements and conversation of historical figures appearing within a virtual reality space.

[1654] "Means for providing a virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with" means means for providing the generated virtual reality space to a user and transmitting data to a headset or other device for the user to experience and interact within the space.

[1655] An "emotion engine" is software that analyzes and recognizes emotions from a user's voice, facial expressions, actions, etc.

[1656] The "means for generating responses of historical figures in real time based on emotion data" is a technology for generating responses and behaviors of historical figures in real time based on the emotion data of a user.

[1657] This invention relates to a virtual reality system that allows users to experience past eras realistically and interact with historical figures. By combining it with an emotion engine, it provides real-time responses and behaviors according to the user's emotions, realizing a more natural and immersive experience.

[1658] System Program Overview

[1659] The server receives requests from users for specific eras or events. For example, if a user requests a "factory tour from the early 20th century," the server analyzes the request and uses generative artificial intelligence to generate a virtual reality space. This virtual reality space realistically reproduces the factory's scenery, the operation of machines, and the behavior of workers.

[1660] The server then simulates the behavior and conversation of historical figures (in this case, factory workers and engineers) within the virtual reality space, enabling realistic interactions within the virtual reality space experienced by the user.

[1661] The generated virtual reality space is provided to the user via a VR headset and controller, allowing the user to experience a past era.

[1662] Hardware and Software Details

[1663] VR headset: A device that provides users with a virtual reality space.

[1664] Robot guide: A robot for interacting with users.

[1665] Emotion recognition sensor: A sensor that acquires emotions from the user's voice, facial expressions, and movements.

[1666] OpenAI GPT-3: As a generative artificial intelligence, it is used to analyze user requests and generate real-time responses based on virtual reality spatial data and emotions.

[1667] Emotion Recognition Library: A library for analyzing user emotions and obtaining emotional data.

[1668] VR Renderer: Rendering software for generating virtual reality spaces.

[1669] Robot Controller software: Software for controlling the operation of the robot guide.

[1670] Specific examples

[1671] For example, if a user requests "I want to experience a factory tour from the early 20th century," the server generates a virtual reality space using the following prompts:

[1672] "A user requests a tour of an early 20th century factory. What kind of scene would you generate?"

[1673] In the generated virtual reality space, old machinery and work scenes in the factory are realistically reproduced. If the user is excited, the emotion recognition sensor will recognize that emotion and the emotion engine will analyze the data. For example, if the user excitedly asks, "How does this machine work?", the robot guide can respond, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[1674] In this way, a system is realized that allows users to experience a past era with a sense of realism while engaging in emotional interaction.

[1675] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1676] Step 1:

[1677] The user inputs a request to experience a specific era or event. Specifically, the user sends a request via their terminal saying, "I would like to experience a factory tour from the early 20th century." This input data is sent to the server, which receives and analyzes the request.

[1678] Step 2:

[1679] Based on the received request, the server generates prompts to generate a virtual reality space using a generative AI model (OpenAI GPT-3). The input is the user's request, and the output is virtual data that defines the specific scenario and components of the virtual reality space. For example, it generates a prompt such as, "The user has requested a factory tour from the early 20th century. What kind of scene would you like to generate?"

[1680] Step 3:

[1681] The server inputs prompts into the generative AI model to generate details of the virtual reality space. The input is the prompt sentence, and the output is detailed virtual reality space data (e.g., factory scenery, machine operation, worker behavior, etc.). The server passes this output to the VR Renderer, which renders the virtual reality space.

[1682] Step 4:

[1683] The terminal receives the rendered virtual reality space data and provides it to the user via a VR headset. The user wears the VR headset and virtually experiences an early 20th century factory. The input is the rendered virtual reality space data, and the output is a virtual environment experienced by the user visually and aurally.

[1684] Step 5:

[1685] The user interacts with the virtual reality space. For example, the user asks, "How does this machine work?" The device acquires the user's voice data and sends it to the server. The input is the user's voice data, and the output is the transmission of the voice data to the server.

[1686] Step 6:

[1687] The server uses an emotion recognition sensor to recognize emotions from the user's voice data. The input is voice data, and the output is emotion data (e.g., excitement, curiosity). The server analyzes this emotion data with an emotion engine.

[1688] Step 7:

[1689] The server uses a generative AI model to generate real-time responses based on the user's emotions, based on the emotion data acquired by the emotion engine. The input is the emotion data and the user's voice question, and the output is an appropriate response (e.g., "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time").

[1690] Step 8:

[1691] The terminal receives response data from the server and responds to the user in real time via the robot guide. The input is the response data, and the output is a voice response to the user. Specifically, the robot guide says, "This machine was a revolutionary device used in the early 20th century, and it greatly improved productivity at the time."

[1692] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1693] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1694] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1695] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1696] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1697] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1698] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1699] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1700] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1701] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1702] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1703] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1704] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1705] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1706] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1707] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1708] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1709] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1710] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1711] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1712] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1713] The following is further disclosed regarding the above embodiment.

[1714] (Claim 1)

[1715] a means for receiving requests from users for specific periods or events;

[1716] A means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request;

[1717] means for simulating the behavior and conversation of historical figures in the virtual reality space;

[1718] means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with;

[1719] means for generating a response in real time to the user's interaction;

[1720] A system including:

[1721] (Claim 2)

[1722] The system according to claim 1, characterized in that the generative artificial intelligence generates a virtual reality space that includes historical background and cultural elements by referring to a historical database.

[1723] (Claim 3)

[1724] 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input.

[1725] "Example 1"

[1726] (Claim 1)

[1727] a means for receiving requests from users for specific periods or events;

[1728] A means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request;

[1729] means for simulating the behavior and conversation of historical figures in the virtual reality space;

[1730] means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with;

[1731] means for monitoring interaction data from users and generating responses in real time;

[1732] means for generating 3D models of historical backgrounds, landscapes, and buildings using the generative artificial intelligence;

[1733] means for transmitting data of the generated virtual reality space to a user device;

[1734] A system including:

[1735] (Claim 2)

[1736] The system according to claim 1, characterized in that the generative artificial intelligence generates a virtual reality space that includes historical background and cultural elements by referring to a historical database.

[1737] (Claim 3)

[1738] 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input.

[1739] "Application Example 1"

[1740] (Claim 1)

[1741] a means for receiving requests from users for specific periods or events;

[1742] A means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request;

[1743] means for simulating the behavior and conversation of historical figures in the virtual reality space;

[1744] means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with;

[1745] means for generating a response in real time to the user's interaction;

[1746] A means for providing an interface for experiencing a virtual reality space using a smartphone and interacting with characters;

[1747] means for analyzing the conversation between the historical figure and the user using a natural language processing model and generating an appropriate response based on the content of the conversation;

[1748] A system including:

[1749] (Claim 2)

[1750] The system according to claim 1, characterized in that the generative artificial intelligence generates a virtual reality space that includes historical background and cultural elements by referring to a historical database.

[1751] (Claim 3)

[1752] 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input.

[1753] "Example 2: Combining Emotion Engines"

[1754] (Claim 1)

[1755] a means for receiving requests from users for specific periods or events;

[1756] A means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request;

[1757] means for simulating the behavior and conversation of historical figures in the virtual reality space;

[1758] means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with;

[1759] The device analyzes the user's voice, facial expressions, and body movements using an emotion engine to acquire emotion data;

[1760] means for generating a real-time response of a historical figure based on the emotion data of the user;

[1761] A system including:

[1762] (Claim 2)

[1763] The system according to claim 1, characterized in that the generative artificial intelligence generates a virtual reality space that includes historical background and cultural elements by referring to a historical database.

[1764] (Claim 3)

[1765] 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input.

[1766] (Claim 4)

[1767] 2. The system according to claim 1, wherein the terminal analyzes the user's emotion data using an emotion engine and generates a real-time response of a historical figure according to the user's emotion.

[1768] "Application example 2 when combining emotion engines"

[1769] (Claim 1)

[1770] a means for receiving requests from users for specific periods or events;

[1771] A means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request;

[1772] means for simulating the behavior and conversation of historical figures in the virtual reality space;

[1773] means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with;

[1774] a means for generating responses of historical figures in real time based on the user's emotion data using an emotion engine that recognizes the user's emotions;

[1775] A system including:

[1776] (Claim 2)

[1777] The system according to claim 1, characterized in that the generative artificial intelligence generates a virtual reality space that includes historical background and cultural elements by referring to a historical database.

[1778] (Claim 3)

[1779] 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input and emotional data. [Explanation of symbols]

[1780] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving requests from users for specific periods or events; a means for generating a virtual reality space that recreates a past era using generative artificial intelligence based on the request; means for simulating the behavior and conversation of historical figures in the virtual reality space; means for providing the virtual reality space to a user and transmitting the virtual reality space to a device for the user to interact with; means for generating a response in real time to the user's interaction; A system including:

2. The system according to claim 1, wherein the generative artificial intelligence generates a virtual reality space including historical background and cultural elements by referring to a historical database.

3. 10. The system of claim 1, wherein the simulation of historical figures' behavior and conversations is adjusted in real time based on user input.

Images