System

Abstract

A system is provided.SOLUTION: A system including a collection means for collecting data from a plurality of data sources including weather data, economic indicators, social statistics, technology trend data, and environmental data, a future prediction means using machine learning models, deep learning, and statistical modeling, a means for incorporating visual, auditory, and tactile elements based on the predicted data to create a high quality virtual reality space, and a means for recreating future events in the virtual reality space where the user specifies specific dates and conditions to experience future events and interact with future people.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, it is extremely important to accurately predict future events and visually experience those scenarios. However, conventional methods do not integrate the data collection, predictive model development, and technology required to realistically reproduce the results. As a result, there is a lack of technology for specifically and interactively recreating future scenarios in virtual reality spaces, making it difficult to effectively experience and predict future events, such as weather changes, economic fluctuations, and social trends. The present invention aims to solve these problems and provide a system for accurately recreating future events in virtual reality spaces. [Means for solving the problem]

[0005] The present invention first provides a data collection means for collecting necessary information from various data sources, such as weather data, economic indicators, social statistics, technology trend data, and environmental data. Next, a future prediction means for analyzing the collected data using a machine learning model and predicting future events is provided. This future prediction means generates different future scenarios using machine learning, deep learning, and statistical modeling techniques. Then, a means for generating a virtual reality space based on the predicted data is provided, generating a high-quality virtual space incorporating visual, auditory, and tactile elements. Furthermore, a means for recreating future events in the virtual reality space is provided, allowing users to experience future events by specifying specific dates and conditions. This also includes a means for interacting with future characters, allowing users to enjoy interactive conversations with the generated future characters. In this way, a system is provided that integrates data collection, future prediction, virtual reality space generation, and future event reproduction, thereby realizing a realistic and interactive future experience.

[0006] A "data collection tool" is a device, system, or method for automatically collecting required data from various data sources.

[0007] A "future prediction tool" is a device, system, or method for predicting future events based on collected data, including the use of techniques such as machine learning and deep learning.

[0008] A "means for generating a virtual reality space" is a device, system, or method for creating a high-quality virtual reality environment by incorporating visual, auditory, tactile, and other elements based on predictive data.

[0009] A "means for recreating future events" is a device, system, or method that allows a user to specify a specific date or conditions and experience a future event in a virtual reality space.

[0010] "Data collection" is the process of obtaining the required information from various data sources.

[0011] "Machine learning" refers to algorithms and techniques used to recognize data patterns and make predictions or classifications.

[0012] "Deep learning" is a type of machine learning that uses multi-layer neural networks to recognize complex data patterns and make predictions.

[0013] "Statistical modeling" is a method of mathematically modeling data relationships to make predictions and perform analysis.

[0014] A "virtual reality space" is a computer-generated, three-dimensional, interactive environment that a user can experience through their senses of sight, hearing, and touch.

[0015] "Specifying a specific date or condition" is the process by which a user selects a future time or situation that interests them and experiences that scenario.

[0016] A "generated future person" is a virtual character generated in a virtual reality space based on predictive data. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that realistically reproduces future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0039] 1. Data Collection

[0040] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute.

[0041] The server formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0042] 2. Future Predictions

[0043] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0044] 3. Generating VR Space

[0045] The terminal generates a virtual reality space based on the predicted data. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. In this virtual space, the user can experience future events in real time.

[0046] 4. Recreating future events

[0047] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0048] For example, if a user specifies a specific future date, they can experience the scenery of Tokyo in 2035. At this time, a person from the future in the virtual space will speak to the user, and the user can enjoy a conversation with that person. This dialogue system uses natural language processing technology to reproduce realistic conversations.

[0049] Specific examples

[0050] 1. Data collection: The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database. Specifically, it retrieves data such as the average temperature, precipitation, and wind speed for each year.

[0051] 2. Future prediction: The server uses machine learning models to predict future weather based on the collected weather data. For example, it uses deep learning models to predict changes in average temperature over the next 10 years.

[0052] 3. VR space generation: The device recreates the city of Tokyo in 2035 based on future weather data generated by a predictive model, including futuristic buildings and transportation as visual elements, and urban noise as auditory elements.

[0053] 4. Recreating future events: Users put on a VR headset and begin a future experience in Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions. They can converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0054] As described above, the system of the present invention integrates data collection, future prediction, VR space generation, and future event reproduction, providing users with the opportunity to experience future events in a realistic and interactive manner.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] The server collects the necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, environmental data, etc. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0058] Step 2:

[0059] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0060] Step 3:

[0061] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0062] Step 4:

[0063] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0064] Step 5:

[0065] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0066] Step 6:

[0067] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0068] Step 7:

[0069] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0070] Step 8:

[0071] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] In conventional future prediction systems, the processes from data collection to prediction and reproduction in virtual reality space were not consistent, and each process was often operated independently. This meant that data processing and integration of predictive models required a great deal of time and effort, making it difficult for users to easily experience future events. Furthermore, when experiencing future scenarios, the system lacked the ability for users to specify specific conditions or dates, and the ability to interact with generative AI models.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the predicted data, a means for recreating future events in the virtual reality space, a means for using a generative AI model for interacting with a person in the future, and a means for allowing the user to specify specific dates and conditions. This enables the system to collect data, make predictions, generate a virtual reality space, and allow the user to interactively experience future events.

[0077] "Data collection means" refers to the means of collecting necessary information from multiple data sources, formatting it, and storing it.

[0078] A "machine learning model" is an algorithm or model used to predict future events using past or present data.

[0079] A "future prediction tool" is a tool that uses machine learning models to analyze data and generate future events and scenarios.

[0080] The "means for generating a virtual reality space" is a means for generating a high-quality virtual reality space based on predicted data.

[0081] The "means for recreating future events in a virtual reality space" refers to a means for recreating predicted future events in a generated virtual reality space.

[0082] A "generative AI model" is a generative artificial intelligence model used to reproduce human conversations and dialogues.

[0083] "Means that allow the user to specify specific dates and conditions" refers to means that the user can input specific dates and conditions via an interface, and a future scenario based on that input is reproduced.

[0084] This invention is a system that realistically recreates future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and the reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0085] 1. Data Collection Methods

[0086] The server collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute. The server then formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0087] Examples:

[0088] The server retrieves weather data from the past 10 years from NASA's weather data API and stores it in a database.

[0089] Obtain global economic indicator data from the API of economic research institutions.

[0090] Collect the latest technology trend data from technology news APIs.

[0091] 2. A means of predicting the future

[0092] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0093] Examples:

[0094] The server uses deep learning models to predict changes in average temperature over the next 10 years.

[0095] Use statistical modeling to predict future economic growth and inflation rates.

[0096] 3. Means of generating VR space

[0097] The terminal generates a virtual reality space based on the prediction data sent from the server. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. Within this virtual space, the user can experience future events in real time.

[0098] Examples:

[0099] The device uses Unity and Unreal Engine to recreate the city of Tokyo in 2035.

[0100] Incorporating futuristic buildings and transportation systems into the VR space.

[0101] 4. A means of recreating future events in a virtual reality space

[0102] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, and this interaction is realized using a generative AI model.

[0103] Examples:

[0104] Users put on a VR headset and begin experiencing the future of Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions.

[0105] Generative AI models can be used to hold natural language conversations with future people.

[0106] Prompt Sentence Examples

[0107] "I want to experience the landscape of Tokyo in 2035"

[0108] "Predict future social trends based on economic indicators for 2030 and recreate that scenario in a virtual reality space."

[0109] Based on the above aspects, the present invention can integrate data collection, future prediction, VR space generation, and future event reproduction. This system allows users to experience future events realistically and interactively.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Step 1:

[0112] Data collection

[0113] The server collects the necessary data from multiple data sources, including weather data, economic indicators, social statistics, technology trend data, and environmental data. It uses various API endpoints and databases as input and obtains formatted data as output.

[0114] Specific behavior:

[0115] The server requests and retrieves data such as average temperature, precipitation, and wind speed for the past 10 years from NASA's weather data API.

[0116] Obtain economic indicator data such as inflation rate, GDP growth rate, and unemployment rate from the API of economic research institutions.

[0117] Collect the latest technology trend data from technology news APIs.

[0118] Step 2:

[0119] Data Formatting and Storage

[0120] The server formats the collected data and converts it into a format suitable for prediction. The formatted data is stored in a future prediction database. Raw data is used as input, and the formatted data is stored in the database as output.

[0121] Specific behavior:

[0122] The server converts the weather data into a format that includes annual average temperature, precipitation, wind speed, etc.

[0123] Convert the economic data into a list and store it indexed by year.

[0124] All formatted data is stored in a NoSQL database.

[0125] Step 3:

[0126] Future predictions

[0127] The server uses the formatted data stored in the database to make future predictions using machine learning models, including deep learning and statistical modeling. The formatted data is used as input, and future prediction data is obtained as output.

[0128] Specific behavior:

[0129] The server loads a deep learning model trained using TensorFlow and performs predictions.

[0130] Predicts average temperature changes and economic growth rates over the next 10 years.

[0131] The prediction results are saved again in the future prediction database.

[0132] Step 4:

[0133] VR space generation

[0134] The device generates a virtual reality space based on future prediction data sent from the server. VR content is generated using software such as Unity or Unreal Engine. Future prediction data is used as input, and a high-quality virtual space is obtained as output.

[0135] Specific behavior:

[0136] The device will create a 3D model based on meteorological and economic data for Tokyo in 2035.

[0137] Unity is used to generate an overall view of a city incorporating futuristic buildings and transportation systems.

[0138] It incorporates visual, auditory, and haptic feedback compatible with VR headsets.

[0139] Step 5:

[0140] Recreating future events

[0141] The device recreates future events in a virtual reality space. Users can specify specific dates and conditions and experience future events generated based on those conditions. Dialogue can be conducted using generative AI models. User-specified conditions and dates are used as input, and a future scenario based on those conditions is reproduced as output.

[0142] Specific behavior:

[0143] The user puts on a VR headset and specifies the weather conditions in Tokyo on August 15, 2035 through the interface.

[0144] The device adjusts the environment and scenario within the virtual space based on the specified conditions.

[0145] Using a generative AI model, we can have natural language conversations with virtual future people.

[0146] By sequentially executing the processing flow of this system's program, users can realistically experience future events based on collected data in a VR space.

[0147] (Application example 1)

[0148] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0149] While technology already exists that can predict future events and recreate them in virtual reality, there is no system that allows users to realistically experience future traffic conditions by applying this technology to driving scenarios for autonomous vehicles. In particular, there is a need for a method to improve the safety of autonomous driving technology and increase user trust by allowing users to experience predicted driving scenarios in virtual reality using future weather and traffic data.

[0150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0151] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, and a means for recreating future driving scenarios of autonomous vehicles in virtual reality based on traffic data. This allows users to realistically experience future road conditions and traffic scenarios, thereby improving reliability of autonomous driving technology.

[0152] "Data collection means" refers to a means that has the function of acquiring and formatting necessary information from various data sources.

[0153] "Methods for predicting the future using machine learning models" are methods for predicting future events based on collected data and generating different future scenarios using machine learning technology.

[0154] The "means for generating a virtual reality space" is a means for creating a high-quality virtual reality space including visual, auditory, and tactile elements based on predictive data.

[0155] The "means for recreating future events in a virtual reality space" refers to a means for allowing a user to experience future events based on specific dates or conditions within a generated virtual reality space.

[0156] "Means for recreating future driving scenarios for autonomous vehicles in virtual reality based on traffic data" refers to a means for predicting future driving scenarios using weather and traffic data, and recreating them as the driving experience of an autonomous vehicle in a virtual reality space.

[0157] This invention is a system that realistically recreates future events in a virtual reality space, and we will explain an embodiment in which it is applied to an autonomous vehicle. The system mainly consists of three entities: a server, a terminal, and a user.

[0158] First, the server collects the necessary information from multiple data sources. Specifically, it collects weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This information is obtained from Web APIs in each field. For example, general weather APIs and traffic information APIs can be used.

[0159] Next, the server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is formatted into a specific format before being stored in the future prediction database. Data processing libraries such as Pandas and NumPy can be used for data formatting.

[0160] The server then uses machine learning models as a means of predicting the future, leveraging machine learning, deep learning, and statistical modeling techniques, such as scikit-learn and TensorFlow, to perform detailed analysis of future events and generate different future scenarios.

[0161] The device generates a virtual reality space based on the predicted data. In this virtual reality space, users can experience future events in real time. Game engines such as Unity and Unreal Engine are used to generate the virtual reality space.

[0162] Furthermore, the device is equipped with a means to recreate future events, allowing users to specify specific dates and conditions. Based on these conditions, future events can be experienced. Users can wear a VR headset and experience road conditions and traffic scenarios in the year 2035, for example, inside an autonomous vehicle.

[0163] For example, the server uses machine learning models to predict weather and traffic conditions for the next 10 years based on weather and traffic data. Based on the predicted data, the device recreates the urban environment of 2035, including future buildings and transportation systems as visual elements and urban noise as auditory elements. Users can experience future driving scenarios using a VR headset.

[0164] An example of a prompt using a generative AI model could be, "We will collect the following data to generate a future predictive scenario: weather data, traffic data. Using the collected data, we will use a machine learning model to predict future weather and traffic conditions for the next five years. Finally, based on the prediction results, we will generate a future driving scenario for an autonomous vehicle in a virtual reality space. In this scenario, users can experience future road conditions using a VR headset."

[0165] As a result, this system integrates everything from predicting future events to generating virtual reality spaces and providing user experiences, making it possible to realistically experience driving scenarios for future autonomous vehicles.

[0166] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0167] Step 1:

[0168] The server collects the necessary information from multiple data sources. Specifically, it acquires weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This data is collected through Web APIs. The input data is raw data collected from each source, and the output data is formatted data. Specifically, the server sends requests to each API and receives responses.

[0169] Step 2:

[0170] The server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is converted into a specific format and then stored in a future prediction database. The input data is the raw data collected in step 1, and the output data is the data converted into an appropriate format. Specifically, the data is formatted using data processing libraries such as Pandas and Numpy.

[0171] Step 3:

[0172] The server uses machine learning models as a means of predicting the future. Based on collected and formatted data, it uses machine learning, deep learning, and statistical modeling techniques to predict future events. The input data is formatted database data, and the output data is a future scenario based on the predictive model. Specifically, it trains and predicts machine learning models using SkitRun and TensorFlow.

[0173] Step 4:

[0174] The device generates a virtual reality space based on the predicted data. The generated virtual reality space provides an environment for the user to experience future events in real time. The input data is a predicted future scenario sent from the server, and the output data is the generated virtual reality space. Specifically, the VR space is constructed using a game engine such as Unity or Unreal Engine.

[0175] Step 5:

[0176] The device is equipped with a means to recreate future events. For this reenactment, the user can specify specific dates and conditions, and experience future events generated based on those conditions. The input data are the conditions and dates specified by the user, and the output data is the corresponding future situation reflected in the virtual reality space. Specifically, the device receives user input and dynamically reconstructs the VR space based on that input.

[0177] Step 6:

[0178] Users use a VR headset to experience future driving scenarios. For example, they can experience realistic road conditions and traffic scenarios in 2035 inside an autonomous vehicle. The input data is the generated VR space, and the output data is the user's experience feedback. Specifically, the user puts on the VR headset and interactively progresses through the experience.

[0179] Through the above steps, the present invention predicts future events and recreates them in a virtual reality space, allowing users to experience the future in a realistic manner. In particular, in the case of autonomous vehicles, it is possible to experience future driving scenarios, which results in improved reliability of the technology.

[0180] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0181] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine.

[0182] 1. Data Collection

[0183] The server first collects the necessary information from multiple data sources. This includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0184] 2. Future Predictions

[0185] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0186] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0187] 3. Generating VR Space

[0188] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0189] 4. Recreating future events

[0190] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0191] 5. Introducing the Emotion Engine

[0192] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0193] 6. Emotion-based interactions

[0194] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0195] Specific examples

[0196] 1. Data Collection:

[0197] The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database, specifically data such as average temperature, precipitation, and wind speed for each year.

[0198] 2. Future Predictions:

[0199] The server uses machine learning models to predict future weather conditions based on the collected weather data, for example, using deep learning models to predict changes in average temperature over the next 10 years.

[0200] 3. VR space generation:

[0201] The device uses future weather data generated by a predictive model to recreate the city of Tokyo in 2035, including future buildings and transportation as visual elements and urban noise as auditory elements.

[0202] 4. Recreating future events:

[0203] Users put on a VR headset and begin their futuristic experience of Tokyo in 2035. They can specify specific dates and weather conditions and experience future scenarios based on those conditions. They can also converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0204] 5. Introducing the Emotion Engine:

[0205] The device captures the user's facial expressions with a camera, and the emotion engine analyzes the facial expression data. For example, if the user smiles, the emotion engine recognizes "joy."

[0206] 6. Emotion-based interactions:

[0207] When the device recognizes the user as "happy," it plays a more positive scenario in the virtual reality space. For example, it displays a scene of a sunny day and a walk in a park in the future. If the user is surprised, it switches to a more relaxing scene.

[0208] In this way, the present invention provides a system that integrates data collection, future prediction, virtual reality space generation, future event reproduction, and dynamic interaction adjustment based on user emotions, thereby providing users with a realistic and interactive future experience.

[0209] The processing flow will be explained below.

[0210] Step 1:

[0211] The server collects necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, and environmental data. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0212] Step 2:

[0213] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training and test data. Then, using deep learning or LSTM models, a model is generated that can predict future events.

[0214] Step 3:

[0215] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0216] Step 4:

[0217] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0218] Step 5:

[0219] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0220] Step 6:

[0221] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0222] Step 7:

[0223] The device activates an emotion engine to recognize the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0224] Step 8:

[0225] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data recognized by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0226] Step 9:

[0227] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0228] Step 10:

[0229] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0230] Example 2

[0231] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] With current technology, it is difficult to allow users to experience future events realistically. Furthermore, there is a lack of interactive systems that dynamically adjust the experience based on the user's emotions. Therefore, there is a need for a system that allows users to realistically understand future events and receive an optimal experience based on their emotions.

[0233] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0234] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for reproducing future events in the virtual reality space, a means for recognizing a user's emotions, and a means for dynamically adjusting a scenario and environment in the virtual reality space based on the emotions, thereby enabling a user to realistically experience future events and optimally interact with them according to their emotions.

[0235] "Data collection means" is a function for collecting necessary information from multiple data sources.

[0236] "Means for predicting the future using machine learning models" is a function for predicting the future using machine learning algorithms based on collected data.

[0237] "Means for generating virtual reality space" refers to a function for creating a virtual reality (VR) environment using computer graphics, etc., based on future prediction data.

[0238] "Means for recreating future events in a virtual reality space" refers to a function for recreating predicted future events or situations as a simulation in the generated virtual reality space.

[0239] The "means for recognizing the user's emotions" is a function for identifying the user's emotional state by analyzing the user's facial expressions, voice, bio-signals, etc.

[0240] "Means for dynamically adjusting the scenario and environment within a virtual reality space based on emotions" is a function for changing the settings and scenario within a virtual reality space in real time based on recognized user emotional data.

[0241] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine. A specific embodiment of this system is shown below.

[0242] The program of this system performs the following processes.

[0243] Data collection

[0244] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected through Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0245] Future predictions

[0246] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that predicts future events. The server also uses the predictive model to generate future scenarios and stores the results in a database. For example, it generates forecast data for temperature and precipitation over the next 10 years.

[0247] VR space generation

[0248] The device generates a virtual reality space based on the predicted data. Based on the predicted data, a high-quality VR environment is created that incorporates visual, auditory, and tactile elements. For example, the cityscape of Tokyo in 2035 is reproduced, and future buildings and traffic conditions are displayed. A VR engine such as Unity is used to generate the virtual reality space.

[0249] Recreating future events

[0250] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, which is realized using a generative AI model. For example, users can experience the streets of Tokyo on August 10, 2035, and the events that will take place on that day.

[0251] Introducing the Emotion Engine

[0252] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0253] Emotion-Based Interaction

[0254] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0255] Through this system, users can realistically experience future events and obtain optimal interactions depending on their emotions. An example of a specific prompt is, "Describe a system that generates a virtual reality space that recreates the streets of Tokyo in the future in 2035, and allows users to experience future events within it by specifying a specific date and weather conditions. Also, describe in detail the mechanism by which the system recognizes the user's emotions during the experience and dynamically adjusts the scenario and environment."

[0256] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0257] Step 1: Data collection

[0258] The server collects the necessary information from multiple data sources. As input, it obtains weather data, economic indicators, social statistics, technology trend data, and environmental data from the respective APIs and databases. Specifically, it sets the API key and endpoint URL and sends an API request. As output, the various types of data obtained are obtained in JSON format, etc., which is then formatted and stored in the database.

[0259] Step 2: Data Shaping

[0260] The server formats the collected data. It receives weather data in JSON format as input. Specifically, it parses the JSON data, extracts necessary fields (e.g., average temperature, precipitation, wind speed, etc.), and structures them. The formatted data is obtained as output, and is stored in a database.

[0261] Step 3: Building a future prediction model

[0262] The server builds a future prediction model based on the formatted data. The input is weather data from the past 10 years obtained from the database. Specifically, the data is scaled and divided into training data and test data. Training is performed using deep learning and LSTM models. The output is a trained future prediction model.

[0263] Step 4: Generate future scenarios

[0264] The server generates future scenarios using the constructed future prediction model. As input, the trained model and the period to be predicted (for example, 10 years into the future) are set. Specifically, input data is given to the prediction model to simulate future weather conditions, economic indicators, etc. The output is predicted temperature and precipitation data, which is stored in a database.

[0265] Step 5: Creating the VR space

[0266] The device generates a virtual reality space based on the prediction data obtained from the server. The prediction data and a VR engine (e.g., Unity) are used as input. Specifically, the system generates 3D models of visual elements such as a virtual Tokyo cityscape, future buildings, and transportation. It also creates auditory elements such as city noise and traffic sounds. The output is a high-quality VR environment.

[0267] Step 6: Recreate future events

[0268] The device recreates future events in a virtual reality space. It receives input such as dates and weather conditions specified by the user. Specifically, it uses a generative AI model to generate a future scenario based on the set conditions. The output is a future scenario that corresponds to the conditions set by the user, recreated in the VR space.

[0269] Step 7: Implementing the Emotion Engine

[0270] The device uses an emotion engine to recognize the user's emotions. It receives the user's facial expression data, biometric signals, and voice data as input. Specifically, it captures facial expressions with a camera and performs voice analysis. It also uses sensors to acquire biometric signals such as heart rate and skin potential. The user's emotional state (happiness, sadness, surprise, anger, etc.) is analyzed as output.

[0271] Step 8: Adjust your interactions based on emotion

[0272] The device dynamically adjusts the scenario and environment in the virtual reality space based on the user's emotional data identified by the emotion engine. The identified user emotional data is used as input. Specifically, if the user is surprised, the environment in the virtual space is changed to a relaxing scene according to that emotion. If the user is happy, a more positive scenario is displayed. The output is an interactive VR experience optimized for the user's emotions.

[0273] (Application example 2)

[0274] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] While existing systems for recreating future events in virtual reality spaces lack the ability to recognize user emotions and dynamically adjust the experience, this has made it difficult to provide personalized interactive experiences. Realizing emotion-based interactions for specific scenarios, such as future shopping experiences, is also a challenge. Furthermore, existing systems have difficulty integrating information from diverse data sources, limiting their ability to make more realistic predictions of the future.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0277] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, an emotion recognition means, and a means for dynamically adjusting a scenario or environment in the virtual reality space based on the emotion data. This makes it possible to recognize a user's emotions in real time and dynamically adjust the experience in the virtual reality space in accordance with the emotions. Furthermore, by collecting data from multiple data sources including consumer behavior data and predicting future consumer behavior and purchasing trends, it is possible to provide a more realistic and personalized future shopping experience.

[0278] "Data collection means" refers to the equipment and functionality for collecting necessary information from multiple data sources.

[0279] "Machine learning models" refer to algorithms and techniques used to predict the future based on collected data.

[0280] "Virtual reality space" refers to a three-dimensional digital environment that users can experience virtually.

[0281] "Means for recreating future events" refers to devices and functions for experiencing events or scenarios that may occur in the future within a virtual reality space.

[0282] "Emotion recognition means" refers to a device and function for identifying the user's emotional state by analyzing the user's facial expressions, voice, and biological signals.

[0283] "Dynamic adjustment means based on emotional data" refers to devices and functions for changing experiences and scenarios within a virtual reality space in real time according to the user's emotional state.

[0284] "Consumer behavior data" refers to information about what products consumers choose and how they behave when purchasing.

[0285] "Personalized interactive experience" refers to providing an experience optimized according to the attributes and emotions of each individual user.

[0286] The system for implementing the present invention includes the following programs: The system is composed of three main entities: a server, a terminal, and a user.

[0287] Data collection methods

[0288] The server first collects the necessary information from multiple data sources. Specifically, it uses Web APIs for each field to collect weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data. The collected data is formatted and stored in a database. For example, the server uses a standard API to obtain consumer behavior data from the past 10 years, organizes it by data item, and registers it in the database.

[0289] Future prediction methods using machine learning models

[0290] The server uses the collected data to train a machine learning model. This model uses deep learning and LSTM (long short-term memory) models. The data is scaled and split into training and test data. The trained model is used to predict future scenarios, and the results are stored in a database. Predictions of future consumer behavior and purchasing trends are generated, allowing for a concrete recreation of the customer experience in the virtual store.

[0291] A means of generating virtual reality space

[0292] The device generates a high-quality virtual reality space based on the prediction data. For this purpose, VR development environments such as Unity and Unreal Engine are used. Based on the prediction data, an interactive VR environment incorporating visual, auditory, and tactile elements is created. For example, future shopping mall and store layouts, new product displays, and interactive try-on experiences can be recreated.

[0293] A means of recreating future events in a virtual reality space

[0294] Users put on a VR headset to experience a virtual store of the future. By specifying a specific date and weather conditions, they can experience future scenarios based on those conditions. Users can participate in future exhibitions and promotional events and experience products and services. During this time, they can also interact with future people through generative AI models.

[0295] emotion recognition means

[0296] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's facial expressions, voice, and biometric signals to identify emotions. For example, if the user smiles, the emotion engine recognizes this as "joy."

[0297] Dynamic adjustment method based on emotion data

[0298] The device dynamically adjusts the experience in the virtual reality space according to the user's emotional state. The scenario and environment can be changed in real time based on the user's emotional data. For example, if a user expresses interest in a product, detailed information and usage scenarios can be added. This allows users to enjoy an optimal interactive shopping experience that is tailored to their emotions.

[0299] Examples and prompts

[0300] As a concrete example, imagine a user walking into a shopping mall of the future, browsing the latest fashion items, when suddenly, a new avatar appears, providing details about the item and explaining the fitting scene.

[0301] Example prompt sentence:

[0302] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[0303] This system allows users to enjoy a futuristic shopping experience while having an interactive experience that responds to their emotional state.

[0304] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0305] Step 1:

[0306] Data collection

[0307] The server collects weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data from Web APIs in each field. Specifically, it retrieves data from the APIs using the Python library requests and stores the collected data in a database.

[0308] Input: Raw data obtained from various APIs

[0309] Output: The formatted data is saved in the database.

[0310] Specific operation: For example, the server retrieves consumer behavior data from the API for the past 10 years and stores it in MongoDB.

[0311] Step 2:

[0312] Data Preprocessing

[0313] The server scales the collected data and splits it into training and test data. Specifically, it uses Python's Pandas library to shape the data and scikit-learn to scale and split the data.

[0314] Input: Data collected in step 1

[0315] Output: Scaled training and test data

[0316] Specific operations: For example, the server creates a data frame, performs standardization processing, and classifies each data set into a training set and a test set.

[0317] Step 3:

[0318] Training a future prediction model

[0319] The server uses the training data to train a machine learning model. Specifically, it builds and trains a deep learning model (such as an LSTM model) using TensorFlow or Keras.

[0320] Input: Scaled training data from step 2

[0321] Output: A trained future prediction model

[0322] Specific operation: For example, the server trains an LSTM model using the training data to generate a model that predicts purchasing trends for the next 10 years.

[0323] Step 4:

[0324] Generating future prediction data

[0325] The server uses the trained model to predict future scenarios and stores the results in a database.

[0326] Input: trained future prediction model, test data

[0327] Output: Predicted future scenario data

[0328] Specific operation: For example, the server uses the trained model to predict consumer behavior scenarios for 2030 and stores the results in a database.

[0329] Step 5:

[0330] VR space generation

[0331] The device uses predictive data to generate a high-quality virtual reality space, specifically, an interactive VR environment incorporating visual, auditory, and tactile elements using VR development environments such as Unity or Unreal Engine.

[0332] Input: Future scenario data generated in step 4

[0333] Output: Generated virtual reality space

[0334] Specific operations: For example, the device can recreate a shopping mall in 2030, creating futuristic store layouts and interactive product displays.

[0335] Step 6:

[0336] Providing future experiences

[0337] Users put on a VR headset and enter a virtual store experience of the future, experiencing scenarios based on specified dates and conditions.

[0338] Input: User-specified conditions (e.g., date, time zone, weather)

[0339] Output: Experience future scenarios based on conditions

[0340] Specific actions: For example, a user puts on a VR headset, visits a shopping mall in 2030, and tries on new products.

[0341] Step 7:

[0342] emotion recognition

[0343] The device monitors the user's facial expressions and voice and uses an emotion engine to recognize emotions in real time. Specifically, it analyzes the user's face using OpenCV and dlib libraries.

[0344] Input: User's facial image and voice data

[0345] Output: Recognized emotional state of the user

[0346] Specific operations: For example, the device captures the user's facial expressions with a camera and uses an emotion engine to recognize "happiness" or "surprise."

[0347] Step 8:

[0348] Dynamic adjustment based on emotion data

[0349] The device adjusts the scenario and environment in the virtual reality space in real time according to the user's emotional state.

[0350] Input: Emotion data recognized in step 7

[0351] Output: A tailored virtual reality experience

[0352] Specific actions: For example, if the user is surprised, switch to a more relaxing environment. Also, display additional details about products that the user is interested in.

[0353] Example: Prompt sentence

[0354] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[0355] These processing steps allow the user to enjoy a future shopping experience while having an interactive experience that is tailored to their emotional state.

[0356] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0357] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0358] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0359] [Second embodiment]

[0360] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0361] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0362] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0363] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0364] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0365] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0366] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0367] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0368] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0369] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0370] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0371] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0372] This invention is a system that realistically reproduces future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0373] 1. Data Collection

[0374] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute.

[0375] The server formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0376] 2. Future Predictions

[0377] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0378] 3. Generating VR Space

[0379] The terminal generates a virtual reality space based on the predicted data. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. In this virtual space, the user can experience future events in real time.

[0380] 4. Recreating future events

[0381] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0382] For example, if a user specifies a specific future date, they can experience the scenery of Tokyo in 2035. At this time, a person from the future in the virtual space will speak to the user, and the user can enjoy a conversation with that person. This dialogue system uses natural language processing technology to reproduce realistic conversations.

[0383] Specific examples

[0384] 1. Data collection: The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database. Specifically, it retrieves data such as the average temperature, precipitation, and wind speed for each year.

[0385] 2. Future prediction: The server uses machine learning models to predict future weather based on the collected weather data. For example, it uses deep learning models to predict changes in average temperature over the next 10 years.

[0386] 3. VR space generation: The device recreates the city of Tokyo in 2035 based on future weather data generated by a predictive model, including futuristic buildings and transportation as visual elements, and urban noise as auditory elements.

[0387] 4. Recreating future events: Users put on a VR headset and begin a future experience in Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions. They can converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0388] As described above, the system of the present invention integrates data collection, future prediction, VR space generation, and future event reproduction, providing users with the opportunity to experience future events in a realistic and interactive manner.

[0389] The processing flow will be explained below.

[0390] Step 1:

[0391] The server collects the necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, environmental data, etc. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0392] Step 2:

[0393] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0394] Step 3:

[0395] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0396] Step 4:

[0397] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0398] Step 5:

[0399] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0400] Step 6:

[0401] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0402] Step 7:

[0403] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0404] Step 8:

[0405] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0406] Example 1

[0407] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0408] In conventional future prediction systems, the processes from data collection to prediction and reproduction in virtual reality space were not consistent, and each process was often operated independently. This meant that data processing and integration of predictive models required a great deal of time and effort, making it difficult for users to easily experience future events. Furthermore, when experiencing future scenarios, the system lacked the ability for users to specify specific conditions or dates, and the ability to interact with generative AI models.

[0409] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0410] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the predicted data, a means for recreating future events in the virtual reality space, a means for using a generative AI model for interacting with a person in the future, and a means for allowing the user to specify specific dates and conditions. This enables the system to collect data, make predictions, generate a virtual reality space, and allow the user to interactively experience future events.

[0411] "Data collection means" refers to the means of collecting necessary information from multiple data sources, formatting it, and storing it.

[0412] A "machine learning model" is an algorithm or model used to predict future events using past or present data.

[0413] A "future prediction tool" is a tool that uses machine learning models to analyze data and generate future events and scenarios.

[0414] The "means for generating a virtual reality space" is a means for generating a high-quality virtual reality space based on predicted data.

[0415] The "means for recreating future events in a virtual reality space" refers to a means for recreating predicted future events in a generated virtual reality space.

[0416] A "generative AI model" is a generative artificial intelligence model used to reproduce human conversations and dialogues.

[0417] "Means that allow the user to specify specific dates and conditions" refers to means that the user can input specific dates and conditions via an interface, and a future scenario based on that input is reproduced.

[0418] This invention is a system that realistically recreates future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and the reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0419] 1. Data Collection Methods

[0420] The server collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute. The server then formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0421] Examples:

[0422] The server retrieves weather data from the past 10 years from NASA's weather data API and stores it in a database.

[0423] Obtain global economic indicator data from the API of economic research institutions.

[0424] Collect the latest technology trend data from technology news APIs.

[0425] 2. A means of predicting the future

[0426] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0427] Examples:

[0428] The server uses deep learning models to predict changes in average temperature over the next 10 years.

[0429] Use statistical modeling to predict future economic growth and inflation rates.

[0430] 3. Means of generating VR space

[0431] The terminal generates a virtual reality space based on the prediction data sent from the server. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. Within this virtual space, the user can experience future events in real time.

[0432] Examples:

[0433] The device uses Unity and Unreal Engine to recreate the city of Tokyo in 2035.

[0434] Incorporating futuristic buildings and transportation systems into the VR space.

[0435] 4. A means of recreating future events in a virtual reality space

[0436] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, and this interaction is realized using a generative AI model.

[0437] Examples:

[0438] Users put on a VR headset and begin experiencing the future of Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions.

[0439] Generative AI models can be used to hold natural language conversations with future people.

[0440] Prompt Sentence Examples

[0441] "I want to experience the landscape of Tokyo in 2035"

[0442] "Predict future social trends based on economic indicators for 2030 and recreate that scenario in a virtual reality space."

[0443] Based on the above aspects, the present invention can integrate data collection, future prediction, VR space generation, and future event reproduction. This system allows users to experience future events realistically and interactively.

[0444] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0445] Step 1:

[0446] Data collection

[0447] The server collects the necessary data from multiple data sources, including weather data, economic indicators, social statistics, technology trend data, and environmental data. It uses various API endpoints and databases as input and obtains formatted data as output.

[0448] Specific behavior:

[0449] The server requests and retrieves data such as average temperature, precipitation, and wind speed for the past 10 years from NASA's weather data API.

[0450] Obtain economic indicator data such as inflation rate, GDP growth rate, and unemployment rate from the API of economic research institutions.

[0451] Collect the latest technology trend data from technology news APIs.

[0452] Step 2:

[0453] Data Formatting and Storage

[0454] The server formats the collected data and converts it into a format suitable for prediction. The formatted data is stored in a future prediction database. Raw data is used as input, and the formatted data is stored in the database as output.

[0455] Specific behavior:

[0456] The server converts the weather data into a format that includes annual average temperature, precipitation, wind speed, etc.

[0457] Convert the economic data into a list and store it indexed by year.

[0458] All formatted data is stored in a NoSQL database.

[0459] Step 3:

[0460] Future predictions

[0461] The server uses the formatted data stored in the database to make future predictions using machine learning models, including deep learning and statistical modeling. The formatted data is used as input, and future prediction data is obtained as output.

[0462] Specific behavior:

[0463] The server loads a deep learning model trained using TensorFlow and performs predictions.

[0464] Predicts average temperature changes and economic growth rates over the next 10 years.

[0465] The prediction results are saved again in the future prediction database.

[0466] Step 4:

[0467] VR space generation

[0468] The device generates a virtual reality space based on future prediction data sent from the server. VR content is generated using software such as Unity or Unreal Engine. Future prediction data is used as input, and a high-quality virtual space is obtained as output.

[0469] Specific behavior:

[0470] The device will create a 3D model based on meteorological and economic data for Tokyo in 2035.

[0471] Unity is used to generate an overall view of a city incorporating futuristic buildings and transportation systems.

[0472] It incorporates visual, auditory, and haptic feedback compatible with VR headsets.

[0473] Step 5:

[0474] Recreating future events

[0475] The device recreates future events in a virtual reality space. Users can specify specific dates and conditions and experience future events generated based on those conditions. Dialogue can be conducted using generative AI models. User-specified conditions and dates are used as input, and a future scenario based on those conditions is reproduced as output.

[0476] Specific behavior:

[0477] The user puts on a VR headset and specifies the weather conditions in Tokyo on August 15, 2035 through the interface.

[0478] The device adjusts the environment and scenario within the virtual space based on the specified conditions.

[0479] Using a generative AI model, we can have natural language conversations with virtual future people.

[0480] By sequentially executing the processing flow of this system's program, users can realistically experience future events based on collected data in a VR space.

[0481] (Application example 1)

[0482] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0483] While technology already exists that can predict future events and recreate them in virtual reality, there is no system that allows users to realistically experience future traffic conditions by applying this technology to driving scenarios for autonomous vehicles. In particular, there is a need for a method to improve the safety of autonomous driving technology and increase user trust by allowing users to experience predicted driving scenarios in virtual reality using future weather and traffic data.

[0484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0485] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, and a means for recreating future driving scenarios of autonomous vehicles in virtual reality based on traffic data. This allows users to realistically experience future road conditions and traffic scenarios, thereby improving reliability of autonomous driving technology.

[0486] "Data collection means" refers to a means that has the function of acquiring and formatting necessary information from various data sources.

[0487] "Methods for predicting the future using machine learning models" are methods for predicting future events based on collected data and generating different future scenarios using machine learning technology.

[0488] The "means for generating a virtual reality space" is a means for creating a high-quality virtual reality space including visual, auditory, and tactile elements based on predictive data.

[0489] The "means for recreating future events in a virtual reality space" refers to a means for allowing a user to experience future events based on specific dates or conditions within a generated virtual reality space.

[0490] "Means for recreating future driving scenarios for autonomous vehicles in virtual reality based on traffic data" refers to a means for predicting future driving scenarios using weather and traffic data, and recreating them as the driving experience of an autonomous vehicle in a virtual reality space.

[0491] This invention is a system that realistically recreates future events in a virtual reality space, and we will explain an embodiment in which it is applied to an autonomous vehicle. The system mainly consists of three entities: a server, a terminal, and a user.

[0492] First, the server collects the necessary information from multiple data sources. Specifically, it collects weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This information is obtained from Web APIs in each field. For example, general weather APIs and traffic information APIs can be used.

[0493] Next, the server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is formatted into a specific format before being stored in the future prediction database. Data processing libraries such as Pandas and NumPy can be used for data formatting.

[0494] The server then uses machine learning models as a means of predicting the future, leveraging machine learning, deep learning, and statistical modeling techniques, such as scikit-learn and TensorFlow, to perform detailed analysis of future events and generate different future scenarios.

[0495] The device generates a virtual reality space based on the predicted data. In this virtual reality space, users can experience future events in real time. Game engines such as Unity and Unreal Engine are used to generate the virtual reality space.

[0496] Furthermore, the device is equipped with a means to recreate future events, allowing users to specify specific dates and conditions. Based on these conditions, future events can be experienced. Users can wear a VR headset and experience road conditions and traffic scenarios in the year 2035, for example, inside an autonomous vehicle.

[0497] For example, the server uses machine learning models to predict weather and traffic conditions for the next 10 years based on weather and traffic data. Based on the predicted data, the device recreates the urban environment of 2035, including future buildings and transportation systems as visual elements and urban noise as auditory elements. Users can experience future driving scenarios using a VR headset.

[0498] An example of a prompt using a generative AI model could be, "We will collect the following data to generate a future predictive scenario: weather data, traffic data. Using the collected data, we will use a machine learning model to predict future weather and traffic conditions for the next five years. Finally, based on the prediction results, we will generate a future driving scenario for an autonomous vehicle in a virtual reality space. In this scenario, users can experience future road conditions using a VR headset."

[0499] As a result, this system integrates everything from predicting future events to generating virtual reality spaces and providing user experiences, making it possible to realistically experience driving scenarios for future autonomous vehicles.

[0500] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0501] Step 1:

[0502] The server collects the necessary information from multiple data sources. Specifically, it acquires weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This data is collected through Web APIs. The input data is raw data collected from each source, and the output data is formatted data. Specifically, the server sends requests to each API and receives responses.

[0503] Step 2:

[0504] The server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is converted into a specific format and then stored in a future prediction database. The input data is the raw data collected in step 1, and the output data is the data converted into an appropriate format. Specifically, the data is formatted using data processing libraries such as Pandas and Numpy.

[0505] Step 3:

[0506] The server uses machine learning models as a means of predicting the future. Based on collected and formatted data, it uses machine learning, deep learning, and statistical modeling techniques to predict future events. The input data is formatted database data, and the output data is a future scenario based on the predictive model. Specifically, it trains and predicts machine learning models using SkitRun and TensorFlow.

[0507] Step 4:

[0508] The device generates a virtual reality space based on the predicted data. The generated virtual reality space provides an environment for the user to experience future events in real time. The input data is a predicted future scenario sent from the server, and the output data is the generated virtual reality space. Specifically, the VR space is constructed using a game engine such as Unity or Unreal Engine.

[0509] Step 5:

[0510] The device is equipped with a means to recreate future events. For this reenactment, the user can specify specific dates and conditions, and experience future events generated based on those conditions. The input data are the conditions and dates specified by the user, and the output data is the corresponding future situation reflected in the virtual reality space. Specifically, the device receives user input and dynamically reconstructs the VR space based on that input.

[0511] Step 6:

[0512] Users use a VR headset to experience future driving scenarios. For example, they can experience realistic road conditions and traffic scenarios in 2035 inside an autonomous vehicle. The input data is the generated VR space, and the output data is the user's experience feedback. Specifically, the user puts on the VR headset and interactively progresses through the experience.

[0513] Through the above steps, the present invention predicts future events and recreates them in a virtual reality space, allowing users to experience the future in a realistic manner. In particular, in the case of autonomous vehicles, it is possible to experience future driving scenarios, which results in improved reliability of the technology.

[0514] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0515] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine.

[0516] 1. Data Collection

[0517] The server first collects the necessary information from multiple data sources. This includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0518] 2. Future Predictions

[0519] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0520] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0521] 3. Generating VR Space

[0522] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0523] 4. Recreating future events

[0524] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0525] 5. Introducing the Emotion Engine

[0526] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0527] 6. Emotion-based interactions

[0528] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0529] Specific examples

[0530] 1. Data Collection:

[0531] The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database, specifically data such as average temperature, precipitation, and wind speed for each year.

[0532] 2. Future Predictions:

[0533] The server uses machine learning models to predict future weather conditions based on the collected weather data, for example, using deep learning models to predict changes in average temperature over the next 10 years.

[0534] 3. VR space generation:

[0535] The device uses future weather data generated by a predictive model to recreate the city of Tokyo in 2035, including future buildings and transportation as visual elements and urban noise as auditory elements.

[0536] 4. Recreating future events:

[0537] Users put on a VR headset and begin their futuristic experience of Tokyo in 2035. They can specify specific dates and weather conditions and experience future scenarios based on those conditions. They can also converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0538] 5. Introducing the Emotion Engine:

[0539] The device captures the user's facial expressions with a camera, and the emotion engine analyzes the facial expression data. For example, if the user smiles, the emotion engine recognizes "joy."

[0540] 6. Emotion-based interactions:

[0541] When the device recognizes the user as "happy," it plays a more positive scenario in the virtual reality space. For example, it displays a scene of a sunny day and a walk in a park in the future. If the user is surprised, it switches to a more relaxing scene.

[0542] In this way, the present invention provides a system that integrates data collection, future prediction, virtual reality space generation, future event reproduction, and dynamic interaction adjustment based on user emotions, thereby providing users with a realistic and interactive future experience.

[0543] The processing flow will be explained below.

[0544] Step 1:

[0545] The server collects necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, and environmental data. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0546] Step 2:

[0547] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training and test data. Then, using deep learning or LSTM models, a model is generated that can predict future events.

[0548] Step 3:

[0549] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0550] Step 4:

[0551] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0552] Step 5:

[0553] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0554] Step 6:

[0555] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0556] Step 7:

[0557] The device activates an emotion engine to recognize the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0558] Step 8:

[0559] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data recognized by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0560] Step 9:

[0561] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0562] Step 10:

[0563] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0564] Example 2

[0565] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0566] With current technology, it is difficult to allow users to experience future events realistically. Furthermore, there is a lack of interactive systems that dynamically adjust the experience based on the user's emotions. Therefore, there is a need for a system that allows users to realistically understand future events and receive an optimal experience based on their emotions.

[0567] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0568] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for reproducing future events in the virtual reality space, a means for recognizing a user's emotions, and a means for dynamically adjusting a scenario and environment in the virtual reality space based on the emotions, thereby enabling a user to realistically experience future events and optimally interact with them according to their emotions.

[0569] "Data collection means" is a function for collecting necessary information from multiple data sources.

[0570] "Means for predicting the future using machine learning models" is a function for predicting the future using machine learning algorithms based on collected data.

[0571] "Means for generating virtual reality space" refers to a function for creating a virtual reality (VR) environment using computer graphics, etc., based on future prediction data.

[0572] "Means for recreating future events in a virtual reality space" refers to a function for recreating predicted future events or situations as a simulation in the generated virtual reality space.

[0573] The "means for recognizing the user's emotions" is a function for identifying the user's emotional state by analyzing the user's facial expressions, voice, bio-signals, etc.

[0574] "Means for dynamically adjusting the scenario and environment within a virtual reality space based on emotions" is a function for changing the settings and scenario within a virtual reality space in real time based on recognized user emotional data.

[0575] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine. A specific embodiment of this system is shown below.

[0576] The program of this system performs the following processes.

[0577] Data collection

[0578] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected through Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0579] Future predictions

[0580] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that predicts future events. The server also uses the predictive model to generate future scenarios and stores the results in a database. For example, it generates forecast data for temperature and precipitation over the next 10 years.

[0581] VR space generation

[0582] The device generates a virtual reality space based on the predicted data. Based on the predicted data, a high-quality VR environment is created that incorporates visual, auditory, and tactile elements. For example, the cityscape of Tokyo in 2035 is reproduced, and future buildings and traffic conditions are displayed. A VR engine such as Unity is used to generate the virtual reality space.

[0583] Recreating future events

[0584] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, which is realized using a generative AI model. For example, users can experience the streets of Tokyo on August 10, 2035, and the events that will take place on that day.

[0585] Introducing the Emotion Engine

[0586] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0587] Emotion-Based Interaction

[0588] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0589] Through this system, users can realistically experience future events and obtain optimal interactions depending on their emotions. An example of a specific prompt is, "Describe a system that generates a virtual reality space that recreates the streets of Tokyo in the future in 2035, and allows users to experience future events within it by specifying a specific date and weather conditions. Also, describe in detail the mechanism by which the system recognizes the user's emotions during the experience and dynamically adjusts the scenario and environment."

[0590] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0591] Step 1: Data collection

[0592] The server collects the necessary information from multiple data sources. As input, it obtains weather data, economic indicators, social statistics, technology trend data, and environmental data from the respective APIs and databases. Specifically, it sets the API key and endpoint URL and sends an API request. As output, the various types of data obtained are obtained in JSON format, etc., which is then formatted and stored in the database.

[0593] Step 2: Data Shaping

[0594] The server formats the collected data. It receives weather data in JSON format as input. Specifically, it parses the JSON data, extracts necessary fields (e.g., average temperature, precipitation, wind speed, etc.), and structures them. The formatted data is obtained as output, and is stored in a database.

[0595] Step 3: Building a future prediction model

[0596] The server builds a future prediction model based on the formatted data. The input is weather data from the past 10 years obtained from the database. Specifically, the data is scaled and divided into training data and test data. Training is performed using deep learning and LSTM models. The output is a trained future prediction model.

[0597] Step 4: Generate future scenarios

[0598] The server generates future scenarios using the constructed future prediction model. As input, the trained model and the period to be predicted (for example, 10 years into the future) are set. Specifically, input data is given to the prediction model to simulate future weather conditions, economic indicators, etc. The output is predicted temperature and precipitation data, which is stored in a database.

[0599] Step 5: Creating the VR space

[0600] The device generates a virtual reality space based on the prediction data obtained from the server. The prediction data and a VR engine (e.g., Unity) are used as input. Specifically, the system generates 3D models of visual elements such as a virtual Tokyo cityscape, future buildings, and transportation. It also creates auditory elements such as city noise and traffic sounds. The output is a high-quality VR environment.

[0601] Step 6: Recreate future events

[0602] The device recreates future events in a virtual reality space. It receives input such as dates and weather conditions specified by the user. Specifically, it uses a generative AI model to generate a future scenario based on the set conditions. The output is a future scenario that corresponds to the conditions set by the user, recreated in the VR space.

[0603] Step 7: Implementing the Emotion Engine

[0604] The device uses an emotion engine to recognize the user's emotions. It receives the user's facial expression data, biometric signals, and voice data as input. Specifically, it captures facial expressions with a camera and performs voice analysis. It also uses sensors to acquire biometric signals such as heart rate and skin potential. The user's emotional state (happiness, sadness, surprise, anger, etc.) is analyzed as output.

[0605] Step 8: Adjust your interactions based on emotion

[0606] The device dynamically adjusts the scenario and environment in the virtual reality space based on the user's emotional data identified by the emotion engine. The identified user emotional data is used as input. Specifically, if the user is surprised, the environment in the virtual space is changed to a relaxing scene according to that emotion. If the user is happy, a more positive scenario is displayed. The output is an interactive VR experience optimized for the user's emotions.

[0607] (Application example 2)

[0608] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0609] While existing systems for recreating future events in virtual reality spaces lack the ability to recognize user emotions and dynamically adjust the experience, this has made it difficult to provide personalized interactive experiences. Realizing emotion-based interactions for specific scenarios, such as future shopping experiences, is also a challenge. Furthermore, existing systems have difficulty integrating information from diverse data sources, limiting their ability to make more realistic predictions of the future.

[0610] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0611] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, an emotion recognition means, and a means for dynamically adjusting a scenario or environment in the virtual reality space based on the emotion data. This makes it possible to recognize a user's emotions in real time and dynamically adjust the experience in the virtual reality space in accordance with the emotions. Furthermore, by collecting data from multiple data sources including consumer behavior data and predicting future consumer behavior and purchasing trends, it is possible to provide a more realistic and personalized future shopping experience.

[0612] "Data collection means" refers to the equipment and functionality for collecting necessary information from multiple data sources.

[0613] "Machine learning models" refer to algorithms and techniques used to predict the future based on collected data.

[0614] "Virtual reality space" refers to a three-dimensional digital environment that users can experience virtually.

[0615] "Means for recreating future events" refers to devices and functions for experiencing events or scenarios that may occur in the future within a virtual reality space.

[0616] "Emotion recognition means" refers to a device and function for identifying the user's emotional state by analyzing the user's facial expressions, voice, and biological signals.

[0617] "Dynamic adjustment means based on emotional data" refers to devices and functions for changing experiences and scenarios within a virtual reality space in real time according to the user's emotional state.

[0618] "Consumer behavior data" refers to information about what products consumers choose and how they behave when purchasing.

[0619] "Personalized interactive experience" refers to providing an experience optimized according to the attributes and emotions of each individual user.

[0620] The system for implementing the present invention includes the following programs: The system is composed of three main entities: a server, a terminal, and a user.

[0621] Data collection methods

[0622] The server first collects the necessary information from multiple data sources. Specifically, it uses Web APIs for each field to collect weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data. The collected data is formatted and stored in a database. For example, the server uses a standard API to obtain consumer behavior data from the past 10 years, organizes it by data item, and registers it in the database.

[0623] Future prediction methods using machine learning models

[0624] The server uses the collected data to train a machine learning model. This model uses deep learning and LSTM (long short-term memory) models. The data is scaled and split into training and test data. The trained model is used to predict future scenarios, and the results are stored in a database. Predictions of future consumer behavior and purchasing trends are generated, allowing for a concrete recreation of the customer experience in the virtual store.

[0625] A means of generating virtual reality space

[0626] The device generates a high-quality virtual reality space based on the prediction data. For this purpose, VR development environments such as Unity and Unreal Engine are used. Based on the prediction data, an interactive VR environment incorporating visual, auditory, and tactile elements is created. For example, future shopping mall and store layouts, new product displays, and interactive try-on experiences can be recreated.

[0627] A means of recreating future events in a virtual reality space

[0628] Users put on a VR headset to experience a virtual store of the future. By specifying a specific date and weather conditions, they can experience future scenarios based on those conditions. Users can participate in future exhibitions and promotional events and experience products and services. During this time, they can also interact with future people through generative AI models.

[0629] emotion recognition means

[0630] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's facial expressions, voice, and biometric signals to identify emotions. For example, if the user smiles, the emotion engine recognizes this as "joy."

[0631] Dynamic adjustment method based on emotion data

[0632] The device dynamically adjusts the experience in the virtual reality space according to the user's emotional state. The scenario and environment can be changed in real time based on the user's emotional data. For example, if a user expresses interest in a product, detailed information and usage scenarios can be added. This allows users to enjoy an optimal interactive shopping experience that is tailored to their emotions.

[0633] Examples and prompts

[0634] As a concrete example, imagine a user walking into a shopping mall of the future, browsing the latest fashion items, when suddenly, a new avatar appears, providing details about the item and explaining the fitting scene.

[0635] Example prompt sentence:

[0636] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[0637] This system allows users to enjoy a futuristic shopping experience while having an interactive experience that responds to their emotional state.

[0638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0639] Step 1:

[0640] Data collection

[0641] The server collects weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data from Web APIs in each field. Specifically, it retrieves data from the APIs using the Python library requests and stores the collected data in a database.

[0642] Input: Raw data obtained from various APIs

[0643] Output: The formatted data is saved in the database.

[0644] Specific operation: For example, the server retrieves consumer behavior data from the API for the past 10 years and stores it in MongoDB.

[0645] Step 2:

[0646] Data Preprocessing

[0647] The server scales the collected data and splits it into training and test data. Specifically, it uses Python's Pandas library to shape the data and scikit-learn to scale and split the data.

[0648] Input: Data collected in step 1

[0649] Output: Scaled training and test data

[0650] Specific operations: For example, the server creates a data frame, performs standardization processing, and classifies each data set into a training set and a test set.

[0651] Step 3:

[0652] Training a future prediction model

[0653] The server uses the training data to train a machine learning model. Specifically, it builds and trains a deep learning model (such as an LSTM model) using TensorFlow or Keras.

[0654] Input: Scaled training data from step 2

[0655] Output: A trained future prediction model

[0656] Specific operation: For example, the server trains an LSTM model using the training data to generate a model that predicts purchasing trends for the next 10 years.

[0657] Step 4:

[0658] Generating future prediction data

[0659] The server uses the trained model to predict future scenarios and stores the results in a database.

[0660] Input: trained future prediction model, test data

[0661] Output: Predicted future scenario data

[0662] Specific operation: For example, the server uses the trained model to predict consumer behavior scenarios for 2030 and stores the results in a database.

[0663] Step 5:

[0664] VR space generation

[0665] The device uses predictive data to generate a high-quality virtual reality space, specifically, an interactive VR environment incorporating visual, auditory, and tactile elements using VR development environments such as Unity or Unreal Engine.

[0666] Input: Future scenario data generated in step 4

[0667] Output: Generated virtual reality space

[0668] Specific operations: For example, the device can recreate a shopping mall in 2030, creating futuristic store layouts and interactive product displays.

[0669] Step 6:

[0670] Providing future experiences

[0671] Users put on a VR headset and enter a virtual store experience of the future, experiencing scenarios based on specified dates and conditions.

[0672] Input: User-specified conditions (e.g., date, time zone, weather)

[0673] Output: Experience future scenarios based on conditions

[0674] Specific actions: For example, a user puts on a VR headset, visits a shopping mall in 2030, and tries on new products.

[0675] Step 7:

[0676] emotion recognition

[0677] The device monitors the user's facial expressions and voice and uses an emotion engine to recognize emotions in real time. Specifically, it analyzes the user's face using OpenCV and dlib libraries.

[0678] Input: User's facial image and voice data

[0679] Output: Recognized emotional state of the user

[0680] Specific operations: For example, the device captures the user's facial expressions with a camera and uses an emotion engine to recognize "happiness" or "surprise."

[0681] Step 8:

[0682] Dynamic adjustment based on emotion data

[0683] The device adjusts the scenario and environment in the virtual reality space in real time according to the user's emotional state.

[0684] Input: Emotion data recognized in step 7

[0685] Output: A tailored virtual reality experience

[0686] Specific actions: For example, if the user is surprised, switch to a more relaxing environment. Also, display additional details about products that the user is interested in.

[0687] Example: Prompt sentence

[0688] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[0689] These processing steps allow the user to enjoy a future shopping experience while having an interactive experience that is tailored to their emotional state.

[0690] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0691] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0692] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0693] [Third embodiment]

[0694] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0695] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0696] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0697] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0698] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0699] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0700] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0701] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0702] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0703] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0704] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0705] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0706] This invention is a system that realistically reproduces future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0707] 1. Data Collection

[0708] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute.

[0709] The server formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0710] 2. Future Predictions

[0711] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0712] 3. Generating VR Space

[0713] The terminal generates a virtual reality space based on the predicted data. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. In this virtual space, the user can experience future events in real time.

[0714] 4. Recreating future events

[0715] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0716] For example, if a user specifies a specific future date, they can experience the scenery of Tokyo in 2035. At this time, a person from the future in the virtual space will speak to the user, and the user can enjoy a conversation with that person. This dialogue system uses natural language processing technology to reproduce realistic conversations.

[0717] Specific examples

[0718] 1. Data collection: The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database. Specifically, it retrieves data such as the average temperature, precipitation, and wind speed for each year.

[0719] 2. Future prediction: The server uses machine learning models to predict future weather based on the collected weather data. For example, it uses deep learning models to predict changes in average temperature over the next 10 years.

[0720] 3. VR space generation: The device recreates the city of Tokyo in 2035 based on future weather data generated by a predictive model, including futuristic buildings and transportation as visual elements, and urban noise as auditory elements.

[0721] 4. Recreating future events: Users put on a VR headset and begin a future experience in Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions. They can converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0722] As described above, the system of the present invention integrates data collection, future prediction, VR space generation, and future event reproduction, providing users with the opportunity to experience future events in a realistic and interactive manner.

[0723] The processing flow will be explained below.

[0724] Step 1:

[0725] The server collects the necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, environmental data, etc. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0726] Step 2:

[0727] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0728] Step 3:

[0729] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0730] Step 4:

[0731] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0732] Step 5:

[0733] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0734] Step 6:

[0735] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0736] Step 7:

[0737] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0738] Step 8:

[0739] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0740] Example 1

[0741] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0742] In conventional future prediction systems, the processes from data collection to prediction and reproduction in virtual reality space were not consistent, and each process was often operated independently. This meant that data processing and integration of predictive models required a great deal of time and effort, making it difficult for users to easily experience future events. Furthermore, when experiencing future scenarios, the system lacked the ability for users to specify specific conditions or dates, and the ability to interact with generative AI models.

[0743] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0744] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the predicted data, a means for recreating future events in the virtual reality space, a means for using a generative AI model for interacting with a person in the future, and a means for allowing the user to specify specific dates and conditions. This enables the system to collect data, make predictions, generate a virtual reality space, and allow the user to interactively experience future events.

[0745] "Data collection means" refers to the means of collecting necessary information from multiple data sources, formatting it, and storing it.

[0746] A "machine learning model" is an algorithm or model used to predict future events using past or present data.

[0747] A "future prediction tool" is a tool that uses machine learning models to analyze data and generate future events and scenarios.

[0748] The "means for generating a virtual reality space" is a means for generating a high-quality virtual reality space based on predicted data.

[0749] The "means for recreating future events in a virtual reality space" refers to a means for recreating predicted future events in a generated virtual reality space.

[0750] A "generative AI model" is a generative artificial intelligence model used to reproduce human conversations and dialogues.

[0751] "Means that allow the user to specify specific dates and conditions" refers to means that the user can input specific dates and conditions via an interface, and a future scenario based on that input is reproduced.

[0752] This invention is a system that realistically recreates future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and the reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[0753] 1. Data Collection Methods

[0754] The server collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute. The server then formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[0755] Examples:

[0756] The server retrieves weather data from the past 10 years from NASA's weather data API and stores it in a database.

[0757] Obtain global economic indicator data from the API of economic research institutions.

[0758] Collect the latest technology trend data from technology news APIs.

[0759] 2. A means of predicting the future

[0760] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[0761] Examples:

[0762] The server uses deep learning models to predict changes in average temperature over the next 10 years.

[0763] Use statistical modeling to predict future economic growth and inflation rates.

[0764] 3. Means of generating VR space

[0765] The terminal generates a virtual reality space based on the prediction data sent from the server. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. Within this virtual space, the user can experience future events in real time.

[0766] Examples:

[0767] The device uses Unity and Unreal Engine to recreate the city of Tokyo in 2035.

[0768] Incorporating futuristic buildings and transportation systems into the VR space.

[0769] 4. A means of recreating future events in a virtual reality space

[0770] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, and this interaction is realized using a generative AI model.

[0771] Examples:

[0772] Users put on a VR headset and begin experiencing the future of Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions.

[0773] Generative AI models can be used to hold natural language conversations with future people.

[0774] Prompt Sentence Examples

[0775] "I want to experience the landscape of Tokyo in 2035"

[0776] "Predict future social trends based on economic indicators for 2030 and recreate that scenario in a virtual reality space."

[0777] Based on the above aspects, the present invention can integrate data collection, future prediction, VR space generation, and future event reproduction. This system allows users to experience future events realistically and interactively.

[0778] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0779] Step 1:

[0780] Data collection

[0781] The server collects the necessary data from multiple data sources, including weather data, economic indicators, social statistics, technology trend data, and environmental data. It uses various API endpoints and databases as input and obtains formatted data as output.

[0782] Specific behavior:

[0783] The server requests and retrieves data such as average temperature, precipitation, and wind speed for the past 10 years from NASA's weather data API.

[0784] Obtain economic indicator data such as inflation rate, GDP growth rate, and unemployment rate from the API of economic research institutions.

[0785] Collect the latest technology trend data from technology news APIs.

[0786] Step 2:

[0787] Data Formatting and Storage

[0788] The server formats the collected data and converts it into a format suitable for prediction. The formatted data is stored in a future prediction database. Raw data is used as input, and the formatted data is stored in the database as output.

[0789] Specific behavior:

[0790] The server converts the weather data into a format that includes annual average temperature, precipitation, wind speed, etc.

[0791] Convert the economic data into a list and store it indexed by year.

[0792] All formatted data is stored in a NoSQL database.

[0793] Step 3:

[0794] Future predictions

[0795] The server uses the formatted data stored in the database to make future predictions using machine learning models, including deep learning and statistical modeling. The formatted data is used as input, and future prediction data is obtained as output.

[0796] Specific behavior:

[0797] The server loads a deep learning model trained using TensorFlow and performs predictions.

[0798] Predicts average temperature changes and economic growth rates over the next 10 years.

[0799] The prediction results are saved again in the future prediction database.

[0800] Step 4:

[0801] VR space generation

[0802] The device generates a virtual reality space based on future prediction data sent from the server. VR content is generated using software such as Unity or Unreal Engine. Future prediction data is used as input, and a high-quality virtual space is obtained as output.

[0803] Specific behavior:

[0804] The device will create a 3D model based on meteorological and economic data for Tokyo in 2035.

[0805] Unity is used to generate an overall view of a city incorporating futuristic buildings and transportation systems.

[0806] It incorporates visual, auditory, and haptic feedback compatible with VR headsets.

[0807] Step 5:

[0808] Recreating future events

[0809] The device recreates future events in a virtual reality space. Users can specify specific dates and conditions and experience future events generated based on those conditions. Dialogue can be conducted using generative AI models. User-specified conditions and dates are used as input, and a future scenario based on those conditions is reproduced as output.

[0810] Specific behavior:

[0811] The user puts on a VR headset and specifies the weather conditions in Tokyo on August 15, 2035 through the interface.

[0812] The device adjusts the environment and scenario within the virtual space based on the specified conditions.

[0813] Using a generative AI model, we can have natural language conversations with virtual future people.

[0814] By sequentially executing the processing flow of this system's program, users can realistically experience future events based on collected data in a VR space.

[0815] (Application example 1)

[0816] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0817] While technology already exists that can predict future events and recreate them in virtual reality, there is no system that allows users to realistically experience future traffic conditions by applying this technology to driving scenarios for autonomous vehicles. In particular, there is a need for a method to improve the safety of autonomous driving technology and increase user trust by allowing users to experience predicted driving scenarios in virtual reality using future weather and traffic data.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0819] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, and a means for recreating future driving scenarios of autonomous vehicles in virtual reality based on traffic data. This allows users to realistically experience future road conditions and traffic scenarios, thereby improving reliability of autonomous driving technology.

[0820] "Data collection means" refers to a means that has the function of acquiring and formatting necessary information from various data sources.

[0821] "Methods for predicting the future using machine learning models" are methods for predicting future events based on collected data and generating different future scenarios using machine learning technology.

[0822] The "means for generating a virtual reality space" is a means for creating a high-quality virtual reality space including visual, auditory, and tactile elements based on predictive data.

[0823] The "means for recreating future events in a virtual reality space" refers to a means for allowing a user to experience future events based on specific dates or conditions within a generated virtual reality space.

[0824] "Means for recreating future driving scenarios for autonomous vehicles in virtual reality based on traffic data" refers to a means for predicting future driving scenarios using weather and traffic data, and recreating them as the driving experience of an autonomous vehicle in a virtual reality space.

[0825] This invention is a system that realistically recreates future events in a virtual reality space, and we will explain an embodiment in which it is applied to an autonomous vehicle. The system mainly consists of three entities: a server, a terminal, and a user.

[0826] First, the server collects the necessary information from multiple data sources. Specifically, it collects weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This information is obtained from Web APIs in each field. For example, general weather APIs and traffic information APIs can be used.

[0827] Next, the server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is formatted into a specific format before being stored in the future prediction database. Data processing libraries such as Pandas and NumPy can be used for data formatting.

[0828] The server then uses machine learning models as a means of predicting the future, leveraging machine learning, deep learning, and statistical modeling techniques, such as scikit-learn and TensorFlow, to perform detailed analysis of future events and generate different future scenarios.

[0829] The device generates a virtual reality space based on the predicted data. In this virtual reality space, users can experience future events in real time. Game engines such as Unity and Unreal Engine are used to generate the virtual reality space.

[0830] Furthermore, the device is equipped with a means to recreate future events, allowing users to specify specific dates and conditions. Based on these conditions, future events can be experienced. Users can wear a VR headset and experience road conditions and traffic scenarios in the year 2035, for example, inside an autonomous vehicle.

[0831] For example, the server uses machine learning models to predict weather and traffic conditions for the next 10 years based on weather and traffic data. Based on the predicted data, the device recreates the urban environment of 2035, including future buildings and transportation systems as visual elements and urban noise as auditory elements. Users can experience future driving scenarios using a VR headset.

[0832] An example of a prompt using a generative AI model could be, "We will collect the following data to generate a future predictive scenario: weather data, traffic data. Using the collected data, we will use a machine learning model to predict future weather and traffic conditions for the next five years. Finally, based on the prediction results, we will generate a future driving scenario for an autonomous vehicle in a virtual reality space. In this scenario, users can experience future road conditions using a VR headset."

[0833] As a result, this system integrates everything from predicting future events to generating virtual reality spaces and providing user experiences, making it possible to realistically experience driving scenarios for future autonomous vehicles.

[0834] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0835] Step 1:

[0836] The server collects the necessary information from multiple data sources. Specifically, it acquires weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This data is collected through Web APIs. The input data is raw data collected from each source, and the output data is formatted data. Specifically, the server sends requests to each API and receives responses.

[0837] Step 2:

[0838] The server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is converted into a specific format and then stored in a future prediction database. The input data is the raw data collected in step 1, and the output data is the data converted into an appropriate format. Specifically, the data is formatted using data processing libraries such as Pandas and Numpy.

[0839] Step 3:

[0840] The server uses machine learning models as a means of predicting the future. Based on collected and formatted data, it uses machine learning, deep learning, and statistical modeling techniques to predict future events. The input data is formatted database data, and the output data is a future scenario based on the predictive model. Specifically, it trains and predicts machine learning models using SkitRun and TensorFlow.

[0841] Step 4:

[0842] The device generates a virtual reality space based on the predicted data. The generated virtual reality space provides an environment for the user to experience future events in real time. The input data is a predicted future scenario sent from the server, and the output data is the generated virtual reality space. Specifically, the VR space is constructed using a game engine such as Unity or Unreal Engine.

[0843] Step 5:

[0844] The device is equipped with a means to recreate future events. For this reenactment, the user can specify specific dates and conditions, and experience future events generated based on those conditions. The input data are the conditions and dates specified by the user, and the output data is the corresponding future situation reflected in the virtual reality space. Specifically, the device receives user input and dynamically reconstructs the VR space based on that input.

[0845] Step 6:

[0846] Users use a VR headset to experience future driving scenarios. For example, they can experience realistic road conditions and traffic scenarios in 2035 inside an autonomous vehicle. The input data is the generated VR space, and the output data is the user's experience feedback. Specifically, the user puts on the VR headset and interactively progresses through the experience.

[0847] Through the above steps, the present invention predicts future events and recreates them in a virtual reality space, allowing users to experience the future in a realistic manner. In particular, in the case of autonomous vehicles, it is possible to experience future driving scenarios, which results in improved reliability of the technology.

[0848] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0849] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine.

[0850] 1. Data Collection

[0851] The server first collects the necessary information from multiple data sources. This includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0852] 2. Future Predictions

[0853] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[0854] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0855] 3. Generating VR Space

[0856] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0857] 4. Recreating future events

[0858] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[0859] 5. Introducing the Emotion Engine

[0860] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0861] 6. Emotion-based interactions

[0862] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0863] Specific examples

[0864] 1. Data Collection:

[0865] The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database, specifically data such as average temperature, precipitation, and wind speed for each year.

[0866] 2. Future Predictions:

[0867] The server uses machine learning models to predict future weather conditions based on the collected weather data, for example, using deep learning models to predict changes in average temperature over the next 10 years.

[0868] 3. VR space generation:

[0869] The device uses future weather data generated by a predictive model to recreate the city of Tokyo in 2035, including future buildings and transportation as visual elements and urban noise as auditory elements.

[0870] 4. Recreating future events:

[0871] Users put on a VR headset and begin their futuristic experience of Tokyo in 2035. They can specify specific dates and weather conditions and experience future scenarios based on those conditions. They can also converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[0872] 5. Introducing the Emotion Engine:

[0873] The device captures the user's facial expressions with a camera, and the emotion engine analyzes the facial expression data. For example, if the user smiles, the emotion engine recognizes "joy."

[0874] 6. Emotion-based interactions:

[0875] When the device recognizes the user as "happy," it plays a more positive scenario in the virtual reality space. For example, it displays a scene of a sunny day and a walk in a park in the future. If the user is surprised, it switches to a more relaxing scene.

[0876] In this way, the present invention provides a system that integrates data collection, future prediction, virtual reality space generation, future event reproduction, and dynamic interaction adjustment based on user emotions, thereby providing users with a realistic and interactive future experience.

[0877] The processing flow will be explained below.

[0878] Step 1:

[0879] The server collects necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, and environmental data. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0880] Step 2:

[0881] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training and test data. Then, using deep learning or LSTM models, a model is generated that can predict future events.

[0882] Step 3:

[0883] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[0884] Step 4:

[0885] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[0886] Step 5:

[0887] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[0888] Step 6:

[0889] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[0890] Step 7:

[0891] The device activates an emotion engine to recognize the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0892] Step 8:

[0893] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data recognized by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0894] Step 9:

[0895] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[0896] Step 10:

[0897] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[0898] Example 2

[0899] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0900] With current technology, it is difficult to allow users to experience future events realistically. Furthermore, there is a lack of interactive systems that dynamically adjust the experience based on the user's emotions. Therefore, there is a need for a system that allows users to realistically understand future events and receive an optimal experience based on their emotions.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0902] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for reproducing future events in the virtual reality space, a means for recognizing a user's emotions, and a means for dynamically adjusting a scenario and environment in the virtual reality space based on the emotions, thereby enabling a user to realistically experience future events and optimally interact with them according to their emotions.

[0903] "Data collection means" is a function for collecting necessary information from multiple data sources.

[0904] "Means for predicting the future using machine learning models" is a function for predicting the future using machine learning algorithms based on collected data.

[0905] "Means for generating virtual reality space" refers to a function for creating a virtual reality (VR) environment using computer graphics, etc., based on future prediction data.

[0906] "Means for recreating future events in a virtual reality space" refers to a function for recreating predicted future events or situations as a simulation in the generated virtual reality space.

[0907] The "means for recognizing the user's emotions" is a function for identifying the user's emotional state by analyzing the user's facial expressions, voice, bio-signals, etc.

[0908] "Means for dynamically adjusting the scenario and environment within a virtual reality space based on emotions" is a function for changing the settings and scenario within a virtual reality space in real time based on recognized user emotional data.

[0909] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine. A specific embodiment of this system is shown below.

[0910] The program of this system performs the following processes.

[0911] Data collection

[0912] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected through Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[0913] Future predictions

[0914] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that predicts future events. The server also uses the predictive model to generate future scenarios and stores the results in a database. For example, it generates forecast data for temperature and precipitation over the next 10 years.

[0915] VR space generation

[0916] The device generates a virtual reality space based on the predicted data. Based on the predicted data, a high-quality VR environment is created that incorporates visual, auditory, and tactile elements. For example, the cityscape of Tokyo in 2035 is reproduced, and future buildings and traffic conditions are displayed. A VR engine such as Unity is used to generate the virtual reality space.

[0917] Recreating future events

[0918] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, which is realized using a generative AI model. For example, users can experience the streets of Tokyo on August 10, 2035, and the events that will take place on that day.

[0919] Introducing the Emotion Engine

[0920] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[0921] Emotion-Based Interaction

[0922] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[0923] Through this system, users can realistically experience future events and obtain optimal interactions depending on their emotions. An example of a specific prompt is, "Describe a system that generates a virtual reality space that recreates the streets of Tokyo in the future in 2035, and allows users to experience future events within it by specifying a specific date and weather conditions. Also, describe in detail the mechanism by which the system recognizes the user's emotions during the experience and dynamically adjusts the scenario and environment."

[0924] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0925] Step 1: Data collection

[0926] The server collects the necessary information from multiple data sources. As input, it obtains weather data, economic indicators, social statistics, technology trend data, and environmental data from the respective APIs and databases. Specifically, it sets the API key and endpoint URL and sends an API request. As output, the various types of data obtained are obtained in JSON format, etc., which is then formatted and stored in the database.

[0927] Step 2: Data Shaping

[0928] The server formats the collected data. It receives weather data in JSON format as input. Specifically, it parses the JSON data, extracts necessary fields (e.g., average temperature, precipitation, wind speed, etc.), and structures them. The formatted data is obtained as output, and is stored in a database.

[0929] Step 3: Building a future prediction model

[0930] The server builds a future prediction model based on the formatted data. The input is weather data from the past 10 years obtained from the database. Specifically, the data is scaled and divided into training data and test data. Training is performed using deep learning and LSTM models. The output is a trained future prediction model.

[0931] Step 4: Generate future scenarios

[0932] The server generates future scenarios using the constructed future prediction model. As input, the trained model and the period to be predicted (for example, 10 years into the future) are set. Specifically, input data is given to the prediction model to simulate future weather conditions, economic indicators, etc. The output is predicted temperature and precipitation data, which is stored in a database.

[0933] Step 5: Creating the VR space

[0934] The device generates a virtual reality space based on the prediction data obtained from the server. The prediction data and a VR engine (e.g., Unity) are used as input. Specifically, the system generates 3D models of visual elements such as a virtual Tokyo cityscape, future buildings, and transportation. It also creates auditory elements such as city noise and traffic sounds. The output is a high-quality VR environment.

[0935] Step 6: Recreate future events

[0936] The device recreates future events in a virtual reality space. It receives input such as dates and weather conditions specified by the user. Specifically, it uses a generative AI model to generate a future scenario based on the set conditions. The output is a future scenario that corresponds to the conditions set by the user, recreated in the VR space.

[0937] Step 7: Implementing the Emotion Engine

[0938] The device uses an emotion engine to recognize the user's emotions. It receives the user's facial expression data, biometric signals, and voice data as input. Specifically, it captures facial expressions with a camera and performs voice analysis. It also uses sensors to acquire biometric signals such as heart rate and skin potential. The user's emotional state (happiness, sadness, surprise, anger, etc.) is analyzed as output.

[0939] Step 8: Adjust your interactions based on emotion

[0940] The device dynamically adjusts the scenario and environment in the virtual reality space based on the user's emotional data identified by the emotion engine. The identified user emotional data is used as input. Specifically, if the user is surprised, the environment in the virtual space is changed to a relaxing scene according to that emotion. If the user is happy, a more positive scenario is displayed. The output is an interactive VR experience optimized for the user's emotions.

[0941] (Application example 2)

[0942] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0943] While existing systems for recreating future events in virtual reality spaces lack the ability to recognize user emotions and dynamically adjust the experience, this has made it difficult to provide personalized interactive experiences. Realizing emotion-based interactions for specific scenarios, such as future shopping experiences, is also a challenge. Furthermore, existing systems have difficulty integrating information from diverse data sources, limiting their ability to make more realistic predictions of the future.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0945] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, an emotion recognition means, and a means for dynamically adjusting a scenario or environment in the virtual reality space based on the emotion data. This makes it possible to recognize a user's emotions in real time and dynamically adjust the experience in the virtual reality space in accordance with the emotions. Furthermore, by collecting data from multiple data sources including consumer behavior data and predicting future consumer behavior and purchasing trends, it is possible to provide a more realistic and personalized future shopping experience.

[0946] "Data collection means" refers to the equipment and functionality for collecting necessary information from multiple data sources.

[0947] "Machine learning models" refer to algorithms and techniques used to predict the future based on collected data.

[0948] "Virtual reality space" refers to a three-dimensional digital environment that users can experience virtually.

[0949] "Means for recreating future events" refers to devices and functions for experiencing events or scenarios that may occur in the future within a virtual reality space.

[0950] "Emotion recognition means" refers to a device and function for identifying the user's emotional state by analyzing the user's facial expressions, voice, and biological signals.

[0951] "Dynamic adjustment means based on emotional data" refers to devices and functions for changing experiences and scenarios within a virtual reality space in real time according to the user's emotional state.

[0952] "Consumer behavior data" refers to information about what products consumers choose and how they behave when purchasing.

[0953] "Personalized interactive experience" refers to providing an experience optimized according to the attributes and emotions of each individual user.

[0954] The system for implementing the present invention includes the following programs: The system is composed of three main entities: a server, a terminal, and a user.

[0955] Data collection methods

[0956] The server first collects the necessary information from multiple data sources. Specifically, it uses Web APIs for each field to collect weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data. The collected data is formatted and stored in a database. For example, the server uses a standard API to obtain consumer behavior data from the past 10 years, organizes it by data item, and registers it in the database.

[0957] Future prediction methods using machine learning models

[0958] The server uses the collected data to train a machine learning model. This model uses deep learning and LSTM (long short-term memory) models. The data is scaled and split into training and test data. The trained model is used to predict future scenarios, and the results are stored in a database. Predictions of future consumer behavior and purchasing trends are generated, allowing for a concrete recreation of the customer experience in the virtual store.

[0959] A means of generating virtual reality space

[0960] The device generates a high-quality virtual reality space based on the prediction data. For this purpose, VR development environments such as Unity and Unreal Engine are used. Based on the prediction data, an interactive VR environment incorporating visual, auditory, and tactile elements is created. For example, future shopping mall and store layouts, new product displays, and interactive try-on experiences can be recreated.

[0961] A means of recreating future events in a virtual reality space

[0962] Users put on a VR headset to experience a virtual store of the future. By specifying a specific date and weather conditions, they can experience future scenarios based on those conditions. Users can participate in future exhibitions and promotional events and experience products and services. During this time, they can also interact with future people through generative AI models.

[0963] emotion recognition means

[0964] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's facial expressions, voice, and biometric signals to identify emotions. For example, if the user smiles, the emotion engine recognizes this as "joy."

[0965] Dynamic adjustment method based on emotion data

[0966] The device dynamically adjusts the experience in the virtual reality space according to the user's emotional state. The scenario and environment can be changed in real time based on the user's emotional data. For example, if a user expresses interest in a product, detailed information and usage scenarios can be added. This allows users to enjoy an optimal interactive shopping experience that is tailored to their emotions.

[0967] Examples and prompts

[0968] As a concrete example, imagine a user walking into a shopping mall of the future, browsing the latest fashion items, when suddenly, a new avatar appears, providing details about the item and explaining the fitting scene.

[0969] Example prompt sentence:

[0970] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[0971] This system allows users to enjoy a futuristic shopping experience while having an interactive experience that responds to their emotional state.

[0972] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0973] Step 1:

[0974] Data collection

[0975] The server collects weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data from Web APIs in each field. Specifically, it retrieves data from the APIs using the Python library requests and stores the collected data in a database.

[0976] Input: Raw data obtained from various APIs

[0977] Output: The formatted data is saved in the database.

[0978] Specific operation: For example, the server retrieves consumer behavior data from the API for the past 10 years and stores it in MongoDB.

[0979] Step 2:

[0980] Data Preprocessing

[0981] The server scales the collected data and splits it into training and test data. Specifically, it uses Python's Pandas library to shape the data and scikit-learn to scale and split the data.

[0982] Input: Data collected in step 1

[0983] Output: Scaled training and test data

[0984] Specific operations: For example, the server creates a data frame, performs standardization processing, and classifies each data set into a training set and a test set.

[0985] Step 3:

[0986] Training a future prediction model

[0987] The server uses the training data to train a machine learning model. Specifically, it builds and trains a deep learning model (such as an LSTM model) using TensorFlow or Keras.

[0988] Input: Scaled training data from step 2

[0989] Output: A trained future prediction model

[0990] Specific operation: For example, the server trains an LSTM model using the training data to generate a model that predicts purchasing trends for the next 10 years.

[0991] Step 4:

[0992] Generating future prediction data

[0993] The server uses the trained model to predict future scenarios and stores the results in a database.

[0994] Input: trained future prediction model, test data

[0995] Output: Predicted future scenario data

[0996] Specific operation: For example, the server uses the trained model to predict consumer behavior scenarios for 2030 and stores the results in a database.

[0997] Step 5:

[0998] VR space generation

[0999] The device uses predictive data to generate a high-quality virtual reality space, specifically, an interactive VR environment incorporating visual, auditory, and tactile elements using VR development environments such as Unity or Unreal Engine.

[1000] Input: Future scenario data generated in step 4

[1001] Output: Generated virtual reality space

[1002] Specific operations: For example, the device can recreate a shopping mall in 2030, creating futuristic store layouts and interactive product displays.

[1003] Step 6:

[1004] Providing future experiences

[1005] Users put on a VR headset and enter a virtual store experience of the future, experiencing scenarios based on specified dates and conditions.

[1006] Input: User-specified conditions (e.g., date, time zone, weather)

[1007] Output: Experience future scenarios based on conditions

[1008] Specific actions: For example, a user puts on a VR headset, visits a shopping mall in 2030, and tries on new products.

[1009] Step 7:

[1010] emotion recognition

[1011] The device monitors the user's facial expressions and voice and uses an emotion engine to recognize emotions in real time. Specifically, it analyzes the user's face using OpenCV and dlib libraries.

[1012] Input: User's facial image and voice data

[1013] Output: Recognized emotional state of the user

[1014] Specific operations: For example, the device captures the user's facial expressions with a camera and uses an emotion engine to recognize "happiness" or "surprise."

[1015] Step 8:

[1016] Dynamic adjustment based on emotion data

[1017] The device adjusts the scenario and environment in the virtual reality space in real time according to the user's emotional state.

[1018] Input: Emotion data recognized in step 7

[1019] Output: A tailored virtual reality experience

[1020] Specific actions: For example, if the user is surprised, switch to a more relaxing environment. Also, display additional details about products that the user is interested in.

[1021] Example: Prompt sentence

[1022] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[1023] These processing steps allow the user to enjoy a future shopping experience while having an interactive experience that is tailored to their emotional state.

[1024] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1025] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1026] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1027] [Fourth embodiment]

[1028] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1029] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1031] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1032] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1033] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1035] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1036] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1037] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1039] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1041] This invention is a system that realistically reproduces future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[1042] 1. Data Collection

[1043] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute.

[1044] The server formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[1045] 2. Future Predictions

[1046] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[1047] 3. Generating VR Space

[1048] The terminal generates a virtual reality space based on the predicted data. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. In this virtual space, the user can experience future events in real time.

[1049] 4. Recreating future events

[1050] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[1051] For example, if a user specifies a specific future date, they can experience the scenery of Tokyo in 2035. At this time, a person from the future in the virtual space will speak to the user, and the user can enjoy a conversation with that person. This dialogue system uses natural language processing technology to reproduce realistic conversations.

[1052] Specific examples

[1053] 1. Data collection: The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database. Specifically, it retrieves data such as the average temperature, precipitation, and wind speed for each year.

[1054] 2. Future prediction: The server uses machine learning models to predict future weather based on the collected weather data. For example, it uses deep learning models to predict changes in average temperature over the next 10 years.

[1055] 3. VR space generation: The device recreates the city of Tokyo in 2035 based on future weather data generated by a predictive model, including futuristic buildings and transportation as visual elements, and urban noise as auditory elements.

[1056] 4. Recreating future events: Users put on a VR headset and begin a future experience in Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions. They can converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[1057] As described above, the system of the present invention integrates data collection, future prediction, VR space generation, and future event reproduction, providing users with the opportunity to experience future events in a realistic and interactive manner.

[1058] The processing flow will be explained below.

[1059] Step 1:

[1060] The server collects the necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, environmental data, etc. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[1061] Step 2:

[1062] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[1063] Step 3:

[1064] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[1065] Step 4:

[1066] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[1067] Step 5:

[1068] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[1069] Step 6:

[1070] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[1071] Step 7:

[1072] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[1073] Step 8:

[1074] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[1075] Example 1

[1076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1077] In conventional future prediction systems, the processes from data collection to prediction and reproduction in virtual reality space were not consistent, and each process was often operated independently. This meant that data processing and integration of predictive models required a great deal of time and effort, making it difficult for users to easily experience future events. Furthermore, when experiencing future scenarios, the system lacked the ability for users to specify specific conditions or dates, and the ability to interact with generative AI models.

[1078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1079] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the predicted data, a means for recreating future events in the virtual reality space, a means for using a generative AI model for interacting with a person in the future, and a means for allowing the user to specify specific dates and conditions. This enables the system to collect data, make predictions, generate a virtual reality space, and allow the user to interactively experience future events.

[1080] "Data collection means" refers to the means of collecting necessary information from multiple data sources, formatting it, and storing it.

[1081] A "machine learning model" is an algorithm or model used to predict future events using past or present data.

[1082] A "future prediction tool" is a tool that uses machine learning models to analyze data and generate future events and scenarios.

[1083] The "means for generating a virtual reality space" is a means for generating a high-quality virtual reality space based on predicted data.

[1084] The "means for recreating future events in a virtual reality space" refers to a means for recreating predicted future events in a generated virtual reality space.

[1085] A "generative AI model" is a generative artificial intelligence model used to reproduce human conversations and dialogues.

[1086] "Means that allow the user to specify specific dates and conditions" refers to means that the user can input specific dates and conditions via an interface, and a future scenario based on that input is reproduced.

[1087] This invention is a system that realistically recreates future events in a virtual reality space, integrating data collection, future prediction, VR space generation, and the reproduction of future events. This system involves three entities: a server, a terminal, and a user.

[1088] 1. Data Collection Methods

[1089] The server collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from web APIs in each field. For example, weather data is obtained from a public weather API, and economic indicators are obtained from the API of an economic research institute. The server then formats the collected data and converts it into a format suitable for prediction. After the data is formatted in a specific format, it is stored in a future prediction database.

[1090] Examples:

[1091] The server retrieves weather data from the past 10 years from NASA's weather data API and stores it in a database.

[1092] Obtain global economic indicator data from the API of economic research institutions.

[1093] Collect the latest technology trend data from technology news APIs.

[1094] 2. A means of predicting the future

[1095] The server uses machine learning models to predict the future based on the data stored in the database. This future prediction method utilizes machine learning, deep learning, and statistical modeling techniques to perform detailed analysis of the collected data and predict future events, thereby generating different future scenarios.

[1096] Examples:

[1097] The server uses deep learning models to predict changes in average temperature over the next 10 years.

[1098] Use statistical modeling to predict future economic growth and inflation rates.

[1099] 3. Means of generating VR space

[1100] The terminal generates a virtual reality space based on the prediction data sent from the server. The VR space generation means creates a high-quality virtual space by incorporating visual, auditory, and tactile elements. Within this virtual space, the user can experience future events in real time.

[1101] Examples:

[1102] The device uses Unity and Unreal Engine to recreate the city of Tokyo in 2035.

[1103] Incorporating futuristic buildings and transportation systems into the VR space.

[1104] 4. A means of recreating future events in a virtual reality space

[1105] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, and this interaction is realized using a generative AI model.

[1106] Examples:

[1107] Users put on a VR headset and begin experiencing the future of Tokyo in 2035. They can specify a specific date and weather conditions and experience a future scenario based on those conditions.

[1108] Generative AI models can be used to hold natural language conversations with future people.

[1109] Prompt Sentence Examples

[1110] "I want to experience the landscape of Tokyo in 2035"

[1111] "Predict future social trends based on economic indicators for 2030 and recreate that scenario in a virtual reality space."

[1112] Based on the above aspects, the present invention can integrate data collection, future prediction, VR space generation, and future event reproduction. This system allows users to experience future events realistically and interactively.

[1113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1114] Step 1:

[1115] Data collection

[1116] The server collects the necessary data from multiple data sources, including weather data, economic indicators, social statistics, technology trend data, and environmental data. It uses various API endpoints and databases as input and obtains formatted data as output.

[1117] Specific behavior:

[1118] The server requests and retrieves data such as average temperature, precipitation, and wind speed for the past 10 years from NASA's weather data API.

[1119] Obtain economic indicator data such as inflation rate, GDP growth rate, and unemployment rate from the API of economic research institutions.

[1120] Collect the latest technology trend data from technology news APIs.

[1121] Step 2:

[1122] Data Formatting and Storage

[1123] The server formats the collected data and converts it into a format suitable for prediction. The formatted data is stored in a future prediction database. Raw data is used as input, and the formatted data is stored in the database as output.

[1124] Specific behavior:

[1125] The server converts the weather data into a format that includes annual average temperature, precipitation, wind speed, etc.

[1126] Convert the economic data into a list and store it indexed by year.

[1127] All formatted data is stored in a NoSQL database.

[1128] Step 3:

[1129] Future predictions

[1130] The server uses the formatted data stored in the database to make future predictions using machine learning models, including deep learning and statistical modeling. The formatted data is used as input, and future prediction data is obtained as output.

[1131] Specific behavior:

[1132] The server loads a deep learning model trained using TensorFlow and performs predictions.

[1133] Predicts average temperature changes and economic growth rates over the next 10 years.

[1134] The prediction results are saved again in the future prediction database.

[1135] Step 4:

[1136] VR space generation

[1137] The device generates a virtual reality space based on future prediction data sent from the server. VR content is generated using software such as Unity or Unreal Engine. Future prediction data is used as input, and a high-quality virtual space is obtained as output.

[1138] Specific behavior:

[1139] The device will create a 3D model based on meteorological and economic data for Tokyo in 2035.

[1140] Unity is used to generate an overall view of a city incorporating futuristic buildings and transportation systems.

[1141] It incorporates visual, auditory, and haptic feedback compatible with VR headsets.

[1142] Step 5:

[1143] Recreating future events

[1144] The device recreates future events in a virtual reality space. Users can specify specific dates and conditions and experience future events generated based on those conditions. Dialogue can be conducted using generative AI models. User-specified conditions and dates are used as input, and a future scenario based on those conditions is reproduced as output.

[1145] Specific behavior:

[1146] The user puts on a VR headset and specifies the weather conditions in Tokyo on August 15, 2035 through the interface.

[1147] The device adjusts the environment and scenario within the virtual space based on the specified conditions.

[1148] Using a generative AI model, we can have natural language conversations with virtual future people.

[1149] By sequentially executing the processing flow of this system's program, users can realistically experience future events based on collected data in a VR space.

[1150] (Application example 1)

[1151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1152] While technology already exists that can predict future events and recreate them in virtual reality, there is no system that allows users to realistically experience future traffic conditions by applying this technology to driving scenarios for autonomous vehicles. In particular, there is a need for a method to improve the safety of autonomous driving technology and increase user trust by allowing users to experience predicted driving scenarios in virtual reality using future weather and traffic data.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1154] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, and a means for recreating future driving scenarios of autonomous vehicles in virtual reality based on traffic data. This allows users to realistically experience future road conditions and traffic scenarios, thereby improving reliability of autonomous driving technology.

[1155] "Data collection means" refers to a means that has the function of acquiring and formatting necessary information from various data sources.

[1156] "Methods for predicting the future using machine learning models" are methods for predicting future events based on collected data and generating different future scenarios using machine learning technology.

[1157] The "means for generating a virtual reality space" is a means for creating a high-quality virtual reality space including visual, auditory, and tactile elements based on predictive data.

[1158] The "means for recreating future events in a virtual reality space" refers to a means for allowing a user to experience future events based on specific dates or conditions within a generated virtual reality space.

[1159] "Means for recreating future driving scenarios for autonomous vehicles in virtual reality based on traffic data" refers to a means for predicting future driving scenarios using weather and traffic data, and recreating them as the driving experience of an autonomous vehicle in a virtual reality space.

[1160] This invention is a system that realistically recreates future events in a virtual reality space, and we will explain an embodiment in which it is applied to an autonomous vehicle. The system mainly consists of three entities: a server, a terminal, and a user.

[1161] First, the server collects the necessary information from multiple data sources. Specifically, it collects weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This information is obtained from Web APIs in each field. For example, general weather APIs and traffic information APIs can be used.

[1162] Next, the server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is formatted into a specific format before being stored in the future prediction database. Data processing libraries such as Pandas and NumPy can be used for data formatting.

[1163] The server then uses machine learning models as a means of predicting the future, leveraging machine learning, deep learning, and statistical modeling techniques, such as scikit-learn and TensorFlow, to perform detailed analysis of future events and generate different future scenarios.

[1164] The device generates a virtual reality space based on the predicted data. In this virtual reality space, users can experience future events in real time. Game engines such as Unity and Unreal Engine are used to generate the virtual reality space.

[1165] Furthermore, the device is equipped with a means to recreate future events, allowing users to specify specific dates and conditions. Based on these conditions, future events can be experienced. Users can wear a VR headset and experience road conditions and traffic scenarios in the year 2035, for example, inside an autonomous vehicle.

[1166] For example, the server uses machine learning models to predict weather and traffic conditions for the next 10 years based on weather and traffic data. Based on the predicted data, the device recreates the urban environment of 2035, including future buildings and transportation systems as visual elements and urban noise as auditory elements. Users can experience future driving scenarios using a VR headset.

[1167] An example of a prompt using a generative AI model could be, "We will collect the following data to generate a future predictive scenario: weather data, traffic data. Using the collected data, we will use a machine learning model to predict future weather and traffic conditions for the next five years. Finally, based on the prediction results, we will generate a future driving scenario for an autonomous vehicle in a virtual reality space. In this scenario, users can experience future road conditions using a VR headset."

[1168] As a result, this system integrates everything from predicting future events to generating virtual reality spaces and providing user experiences, making it possible to realistically experience driving scenarios for future autonomous vehicles.

[1169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1170] Step 1:

[1171] The server collects the necessary information from multiple data sources. Specifically, it acquires weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data. This data is collected through Web APIs. The input data is raw data collected from each source, and the output data is formatted data. Specifically, the server sends requests to each API and receives responses.

[1172] Step 2:

[1173] The server formats the collected data and converts it into a format suitable for prediction. At this stage, each piece of data is converted into a specific format and then stored in a future prediction database. The input data is the raw data collected in step 1, and the output data is the data converted into an appropriate format. Specifically, the data is formatted using data processing libraries such as Pandas and Numpy.

[1174] Step 3:

[1175] The server uses machine learning models as a means of predicting the future. Based on collected and formatted data, it uses machine learning, deep learning, and statistical modeling techniques to predict future events. The input data is formatted database data, and the output data is a future scenario based on the predictive model. Specifically, it trains and predicts machine learning models using SkitRun and TensorFlow.

[1176] Step 4:

[1177] The device generates a virtual reality space based on the predicted data. The generated virtual reality space provides an environment for the user to experience future events in real time. The input data is a predicted future scenario sent from the server, and the output data is the generated virtual reality space. Specifically, the VR space is constructed using a game engine such as Unity or Unreal Engine.

[1178] Step 5:

[1179] The device is equipped with a means to recreate future events. For this reenactment, the user can specify specific dates and conditions, and experience future events generated based on those conditions. The input data are the conditions and dates specified by the user, and the output data is the corresponding future situation reflected in the virtual reality space. Specifically, the device receives user input and dynamically reconstructs the VR space based on that input.

[1180] Step 6:

[1181] Users use a VR headset to experience future driving scenarios. For example, they can experience realistic road conditions and traffic scenarios in 2035 inside an autonomous vehicle. The input data is the generated VR space, and the output data is the user's experience feedback. Specifically, the user puts on the VR headset and interactively progresses through the experience.

[1182] Through the above steps, the present invention predicts future events and recreates them in a virtual reality space, allowing users to experience the future in a realistic manner. In particular, in the case of autonomous vehicles, it is possible to experience future driving scenarios, which results in improved reliability of the technology.

[1183] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1184] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine.

[1185] 1. Data Collection

[1186] The server first collects the necessary information from multiple data sources. This includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected from Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[1187] 2. Future Predictions

[1188] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that can predict future events.

[1189] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[1190] 3. Generating VR Space

[1191] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[1192] 4. Recreating future events

[1193] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, they can interact with the generated future characters, and this interaction is realized using generative AI.

[1194] 5. Introducing the Emotion Engine

[1195] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[1196] 6. Emotion-based interactions

[1197] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[1198] Specific examples

[1199] 1. Data Collection:

[1200] The server retrieves weather data from NASA's weather data API for the past 10 years and stores it in a database, specifically data such as average temperature, precipitation, and wind speed for each year.

[1201] 2. Future Predictions:

[1202] The server uses machine learning models to predict future weather conditions based on the collected weather data, for example, using deep learning models to predict changes in average temperature over the next 10 years.

[1203] 3. VR space generation:

[1204] The device uses future weather data generated by a predictive model to recreate the city of Tokyo in 2035, including future buildings and transportation as visual elements and urban noise as auditory elements.

[1205] 4. Recreating future events:

[1206] Users put on a VR headset and begin their futuristic experience of Tokyo in 2035. They can specify specific dates and weather conditions and experience future scenarios based on those conditions. They can also converse with people from the future through generative AI and deepen their understanding of life in the future based on the information provided by the artificial intelligence.

[1207] 5. Introducing the Emotion Engine:

[1208] The device captures the user's facial expressions with a camera, and the emotion engine analyzes the facial expression data. For example, if the user smiles, the emotion engine recognizes "joy."

[1209] 6. Emotion-based interactions:

[1210] When the device recognizes the user as "happy," it plays a more positive scenario in the virtual reality space. For example, it displays a scene of a sunny day and a walk in a park in the future. If the user is surprised, it switches to a more relaxing scene.

[1211] In this way, the present invention provides a system that integrates data collection, future prediction, virtual reality space generation, future event reproduction, and dynamic interaction adjustment based on user emotions, thereby providing users with a realistic and interactive future experience.

[1212] The processing flow will be explained below.

[1213] Step 1:

[1214] The server collects necessary information from multiple data sources, including weather data APIs, economic indicator APIs, social statistical data, technology trend data, and environmental data. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[1215] Step 2:

[1216] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training and test data. Then, using deep learning or LSTM models, a model is generated that can predict future events.

[1217] Step 3:

[1218] The server uses the predictive model to generate future scenarios and stores the results in a database, such as forecast data for temperature and precipitation over the next 10 years.

[1219] Step 4:

[1220] The device generates a virtual reality space based on the predictive data. This creates a high-quality VR environment incorporating visual, auditory, and tactile elements. For example, a Tokyo streetscape from 2035 could be recreated, displaying future buildings and traffic conditions.

[1221] Step 5:

[1222] The user wears a VR headset via a terminal and accesses the VR space. The user operates an interface to specify a specific date and conditions, and selects a future scenario. For example, the user specifies a summer day in Tokyo in 2035.

[1223] Step 6:

[1224] The device recreates future events in a virtual reality space based on a date and conditions specified by the user. The user can experience the environment through sight, sound, and touch. For example, a user can explore a future Tokyo and have interactive conversations with virtual characters from the future.

[1225] Step 7:

[1226] The device activates an emotion engine to recognize the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[1227] Step 8:

[1228] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data recognized by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[1229] Step 9:

[1230] The device uses generative AI to allow interaction with a future person in a virtual reality space. The user converses with the future person using voice or text input, and the generative AI generates a response in real time. For example, the future person might ask, "Would you like to know about future energy technology?" If the user answers "yes," detailed information is provided.

[1231] Step 10:

[1232] Users can experience future events in a VR space and deepen their knowledge of future environments and scenarios. Users can move freely within the virtual reality space and explore future cities and natural environments. For example, users can visit a virtual park and see a new ecosystem that utilizes future technology.

[1233] Example 2

[1234] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1235] With current technology, it is difficult to allow users to experience future events realistically. Furthermore, there is a lack of interactive systems that dynamically adjust the experience based on the user's emotions. Therefore, there is a need for a system that allows users to realistically understand future events and receive an optimal experience based on their emotions.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1237] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for reproducing future events in the virtual reality space, a means for recognizing a user's emotions, and a means for dynamically adjusting a scenario and environment in the virtual reality space based on the emotions, thereby enabling a user to realistically experience future events and optimally interact with them according to their emotions.

[1238] "Data collection means" is a function for collecting necessary information from multiple data sources.

[1239] "Means for predicting the future using machine learning models" is a function for predicting the future using machine learning algorithms based on collected data.

[1240] "Means for generating virtual reality space" refers to a function for creating a virtual reality (VR) environment using computer graphics, etc., based on future prediction data.

[1241] "Means for recreating future events in a virtual reality space" refers to a function for recreating predicted future events or situations as a simulation in the generated virtual reality space.

[1242] The "means for recognizing the user's emotions" is a function for identifying the user's emotional state by analyzing the user's facial expressions, voice, bio-signals, etc.

[1243] "Means for dynamically adjusting the scenario and environment within a virtual reality space based on emotions" is a function for changing the settings and scenario within a virtual reality space in real time based on recognized user emotional data.

[1244] This invention relates to a system that realistically recreates future events in a virtual reality space, recognizes the user's emotions, and dynamically adjusts the experience. This system involves three entities: a server, a terminal, and a user, and is configured by combining an emotion engine. A specific embodiment of this system is shown below.

[1245] The program of this system performs the following processes.

[1246] Data collection

[1247] The server first collects the necessary information from multiple data sources. Specifically, this includes weather data, economic indicators, social statistics, technology trend data, and environmental data. This data is collected through Web APIs in each field. For example, the server obtains weather data from the past 10 years from NASA's weather data API, formats the data, and stores it in a database.

[1248] Future predictions

[1249] The server uses machine learning algorithms to build a future prediction model based on the formatted data. The data is scaled and split into training data and test data. It then trains deep learning or LSTM models to generate a model that predicts future events. The server also uses the predictive model to generate future scenarios and stores the results in a database. For example, it generates forecast data for temperature and precipitation over the next 10 years.

[1250] VR space generation

[1251] The device generates a virtual reality space based on the predicted data. Based on the predicted data, a high-quality VR environment is created that incorporates visual, auditory, and tactile elements. For example, the cityscape of Tokyo in 2035 is reproduced, and future buildings and traffic conditions are displayed. A VR engine such as Unity is used to generate the virtual reality space.

[1252] Recreating future events

[1253] The device is equipped with a means to recreate future events in a virtual reality space. Users can specify specific dates and conditions, and experience future events generated based on those conditions. Furthermore, users can interact with the generated future characters, which is realized using a generative AI model. For example, users can experience the streets of Tokyo on August 10, 2035, and the events that will take place on that day.

[1254] Introducing the Emotion Engine

[1255] The device is equipped with an emotion engine for recognizing the user's emotions. The emotion engine identifies the user's emotional state using voice analysis, facial expression recognition, and biometric signal data. For example, emotions such as joy, sadness, surprise, and anger can be recognized from the user's facial expressions.

[1256] Emotion-Based Interaction

[1257] The device dynamically adjusts the scenario and environment displayed in the virtual reality space based on the user's emotional data identified by the emotion engine. For example, if the user is surprised, the environment in the virtual space changes according to that emotion, and a scenario inducing a relaxed state is displayed.

[1258] Through this system, users can realistically experience future events and obtain optimal interactions depending on their emotions. An example of a specific prompt is, "Describe a system that generates a virtual reality space that recreates the streets of Tokyo in the future in 2035, and allows users to experience future events within it by specifying a specific date and weather conditions. Also, describe in detail the mechanism by which the system recognizes the user's emotions during the experience and dynamically adjusts the scenario and environment."

[1259] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1260] Step 1: Data collection

[1261] The server collects the necessary information from multiple data sources. As input, it obtains weather data, economic indicators, social statistics, technology trend data, and environmental data from the respective APIs and databases. Specifically, it sets the API key and endpoint URL and sends an API request. As output, the various types of data obtained are obtained in JSON format, etc., which is then formatted and stored in the database.

[1262] Step 2: Data Shaping

[1263] The server formats the collected data. It receives weather data in JSON format as input. Specifically, it parses the JSON data, extracts necessary fields (e.g., average temperature, precipitation, wind speed, etc.), and structures them. The formatted data is obtained as output, and is stored in a database.

[1264] Step 3: Building a future prediction model

[1265] The server builds a future prediction model based on the formatted data. The input is weather data from the past 10 years obtained from the database. Specifically, the data is scaled and divided into training data and test data. Training is performed using deep learning and LSTM models. The output is a trained future prediction model.

[1266] Step 4: Generate future scenarios

[1267] The server generates future scenarios using the constructed future prediction model. As input, the trained model and the period to be predicted (for example, 10 years into the future) are set. Specifically, input data is given to the prediction model to simulate future weather conditions, economic indicators, etc. The output is predicted temperature and precipitation data, which is stored in a database.

[1268] Step 5: Creating the VR space

[1269] The device generates a virtual reality space based on the prediction data obtained from the server. The prediction data and a VR engine (e.g., Unity) are used as input. Specifically, the system generates 3D models of visual elements such as a virtual Tokyo cityscape, future buildings, and transportation. It also creates auditory elements such as city noise and traffic sounds. The output is a high-quality VR environment.

[1270] Step 6: Recreate future events

[1271] The device recreates future events in a virtual reality space. It receives input such as dates and weather conditions specified by the user. Specifically, it uses a generative AI model to generate a future scenario based on the set conditions. The output is a future scenario that corresponds to the conditions set by the user, recreated in the VR space.

[1272] Step 7: Implementing the Emotion Engine

[1273] The device uses an emotion engine to recognize the user's emotions. It receives the user's facial expression data, biometric signals, and voice data as input. Specifically, it captures facial expressions with a camera and performs voice analysis. It also uses sensors to acquire biometric signals such as heart rate and skin potential. The user's emotional state (happiness, sadness, surprise, anger, etc.) is analyzed as output.

[1274] Step 8: Adjust your interactions based on emotion

[1275] The device dynamically adjusts the scenario and environment in the virtual reality space based on the user's emotional data identified by the emotion engine. The identified user emotional data is used as input. Specifically, if the user is surprised, the environment in the virtual space is changed to a relaxing scene according to that emotion. If the user is happy, a more positive scenario is displayed. The output is an interactive VR experience optimized for the user's emotions.

[1276] (Application example 2)

[1277] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1278] While existing systems for recreating future events in virtual reality spaces lack the ability to recognize user emotions and dynamically adjust the experience, this has made it difficult to provide personalized interactive experiences. Realizing emotion-based interactions for specific scenarios, such as future shopping experiences, is also a challenge. Furthermore, existing systems have difficulty integrating information from diverse data sources, limiting their ability to make more realistic predictions of the future.

[1279] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1280] In this invention, the server includes a data collection means, a future prediction means using a machine learning model, a means for generating a virtual reality space based on the prediction data, a means for recreating future events in the virtual reality space, an emotion recognition means, and a means for dynamically adjusting a scenario or environment in the virtual reality space based on the emotion data. This makes it possible to recognize a user's emotions in real time and dynamically adjust the experience in the virtual reality space in accordance with the emotions. Furthermore, by collecting data from multiple data sources including consumer behavior data and predicting future consumer behavior and purchasing trends, it is possible to provide a more realistic and personalized future shopping experience.

[1281] "Data collection means" refers to the equipment and functionality for collecting necessary information from multiple data sources.

[1282] "Machine learning models" refer to algorithms and techniques used to predict the future based on collected data.

[1283] "Virtual reality space" refers to a three-dimensional digital environment that users can experience virtually.

[1284] "Means for recreating future events" refers to devices and functions for experiencing events or scenarios that may occur in the future within a virtual reality space.

[1285] "Emotion recognition means" refers to a device and function for identifying the user's emotional state by analyzing the user's facial expressions, voice, and biological signals.

[1286] "Dynamic adjustment means based on emotional data" refers to devices and functions for changing experiences and scenarios within a virtual reality space in real time according to the user's emotional state.

[1287] "Consumer behavior data" refers to information about what products consumers choose and how they behave when purchasing.

[1288] "Personalized interactive experience" refers to providing an experience optimized according to the attributes and emotions of each individual user.

[1289] The system for implementing the present invention includes the following programs: The system is composed of three main entities: a server, a terminal, and a user.

[1290] Data collection methods

[1291] The server first collects the necessary information from multiple data sources. Specifically, it uses Web APIs for each field to collect weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data. The collected data is formatted and stored in a database. For example, the server uses a standard API to obtain consumer behavior data from the past 10 years, organizes it by data item, and registers it in the database.

[1292] Future prediction methods using machine learning models

[1293] The server uses the collected data to train a machine learning model. This model uses deep learning and LSTM (long short-term memory) models. The data is scaled and split into training and test data. The trained model is used to predict future scenarios, and the results are stored in a database. Predictions of future consumer behavior and purchasing trends are generated, allowing for a concrete recreation of the customer experience in the virtual store.

[1294] A means of generating virtual reality space

[1295] The device generates a high-quality virtual reality space based on the prediction data. For this purpose, VR development environments such as Unity and Unreal Engine are used. Based on the prediction data, an interactive VR environment incorporating visual, auditory, and tactile elements is created. For example, future shopping mall and store layouts, new product displays, and interactive try-on experiences can be recreated.

[1296] A means of recreating future events in a virtual reality space

[1297] Users put on a VR headset to experience a virtual store of the future. By specifying a specific date and weather conditions, they can experience future scenarios based on those conditions. Users can participate in future exhibitions and promotional events and experience products and services. During this time, they can also interact with future people through generative AI models.

[1298] emotion recognition means

[1299] The device is equipped with an emotion engine that recognizes the user's emotions in real time. The emotion engine analyzes the user's facial expressions, voice, and biometric signals to identify emotions. For example, if the user smiles, the emotion engine recognizes this as "joy."

[1300] Dynamic adjustment method based on emotion data

[1301] The device dynamically adjusts the experience in the virtual reality space according to the user's emotional state. The scenario and environment can be changed in real time based on the user's emotional data. For example, if a user expresses interest in a product, detailed information and usage scenarios can be added. This allows users to enjoy an optimal interactive shopping experience that is tailored to their emotions.

[1302] Examples and prompts

[1303] As a concrete example, imagine a user walking into a shopping mall of the future, browsing the latest fashion items, when suddenly, a new avatar appears, providing details about the item and explaining the fitting scene.

[1304] Example prompt sentence:

[1305] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[1306] This system allows users to enjoy a futuristic shopping experience while having an interactive experience that responds to their emotional state.

[1307] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1308] Step 1:

[1309] Data collection

[1310] The server collects weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data from Web APIs in each field. Specifically, it retrieves data from the APIs using the Python library requests and stores the collected data in a database.

[1311] Input: Raw data obtained from various APIs

[1312] Output: The formatted data is saved in the database.

[1313] Specific operation: For example, the server retrieves consumer behavior data from the API for the past 10 years and stores it in MongoDB.

[1314] Step 2:

[1315] Data Preprocessing

[1316] The server scales the collected data and splits it into training and test data. Specifically, it uses Python's Pandas library to shape the data and scikit-learn to scale and split the data.

[1317] Input: Data collected in step 1

[1318] Output: Scaled training and test data

[1319] Specific operations: For example, the server creates a data frame, performs standardization processing, and classifies each data set into a training set and a test set.

[1320] Step 3:

[1321] Training a future prediction model

[1322] The server uses the training data to train a machine learning model. Specifically, it builds and trains a deep learning model (such as an LSTM model) using TensorFlow or Keras.

[1323] Input: Scaled training data from step 2

[1324] Output: A trained future prediction model

[1325] Specific operation: For example, the server trains an LSTM model using the training data to generate a model that predicts purchasing trends for the next 10 years.

[1326] Step 4:

[1327] Generating future prediction data

[1328] The server uses the trained model to predict future scenarios and stores the results in a database.

[1329] Input: trained future prediction model, test data

[1330] Output: Predicted future scenario data

[1331] Specific operation: For example, the server uses the trained model to predict consumer behavior scenarios for 2030 and stores the results in a database.

[1332] Step 5:

[1333] VR space generation

[1334] The device uses predictive data to generate a high-quality virtual reality space, specifically, an interactive VR environment incorporating visual, auditory, and tactile elements using VR development environments such as Unity or Unreal Engine.

[1335] Input: Future scenario data generated in step 4

[1336] Output: Generated virtual reality space

[1337] Specific operations: For example, the device can recreate a shopping mall in 2030, creating futuristic store layouts and interactive product displays.

[1338] Step 6:

[1339] Providing future experiences

[1340] Users put on a VR headset and enter a virtual store experience of the future, experiencing scenarios based on specified dates and conditions.

[1341] Input: User-specified conditions (e.g., date, time zone, weather)

[1342] Output: Experience future scenarios based on conditions

[1343] Specific actions: For example, a user puts on a VR headset, visits a shopping mall in 2030, and tries on new products.

[1344] Step 7:

[1345] emotion recognition

[1346] The device monitors the user's facial expressions and voice and uses an emotion engine to recognize emotions in real time. Specifically, it analyzes the user's face using OpenCV and dlib libraries.

[1347] Input: User's facial image and voice data

[1348] Output: Recognized emotional state of the user

[1349] Specific operations: For example, the device captures the user's facial expressions with a camera and uses an emotion engine to recognize "happiness" or "surprise."

[1350] Step 8:

[1351] Dynamic adjustment based on emotion data

[1352] The device adjusts the scenario and environment in the virtual reality space in real time according to the user's emotional state.

[1353] Input: Emotion data recognized in step 7

[1354] Output: A tailored virtual reality experience

[1355] Specific actions: For example, if the user is surprised, switch to a more relaxing environment. Also, display additional details about products that the user is interested in.

[1356] Example: Prompt sentence

[1357] "You enter a virtual store in the future, in 2030, browsing the latest fashion items. Suddenly, a new avatar appears, providing details about the item and showing you how to try it on."

[1358] These processing steps allow the user to enjoy a future shopping experience while having an interactive experience that is tailored to their emotional state.

[1359] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1360] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1361] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1362] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1363] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1364] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1365] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1366] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1367] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1368] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1369] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1370] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1371] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1372] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1373] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1374] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1375] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1376] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1377] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1378] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1379] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1380] The following is further disclosed regarding the above embodiment.

[1381] (Claim 1)

[1382] data collection means;

[1383] A method for predicting the future using machine learning models,

[1384] means for generating a virtual reality space based on the prediction data;

[1385] A means of recreating future events in a virtual reality space;

[1386] A system including:

[1387] (Claim 2)

[1388] 10. The system of claim 1, wherein the data collection means comprises means for collecting data from a plurality of data sources including meteorological data, economic indicators, social statistics, technology trend data, and environmental data.

[1389] (Claim 3)

[1390] 2. The system of claim 1, wherein the future prediction means includes means for predicting future events and generating different future scenarios using machine learning, deep learning, and statistical modeling techniques.

[1391] (Claim 4)

[1392] 2. The system according to claim 1, wherein the means for generating a virtual reality space includes means for incorporating visual, auditory, and tactile elements to generate a high-quality virtual space.

[1393] (Claim 5)

[1394] The system of claim 1, wherein the means for recreating future events in a virtual reality space includes a means for the user to specify a specific date or conditions to experience future events and a means for the user to interact with the generated future characters.

[1395] "Example 1"

[1396] (Claim 1)

[1397] data collection means;

[1398] A method for predicting the future using machine learning models,

[1399] means for generating a virtual reality space based on the prediction data;

[1400] A means of recreating future events in a virtual reality space;

[1401] A means of using generative AI models to interact with future people;

[1402] A means for the user to specify specific dates or conditions;

[1403] A system including:

[1404] (Claim 2)

[1405] 10. The system of claim 1, wherein the data collection means comprises means for collecting data from a plurality of data sources including meteorological data, economic indicators, social statistics, technology trend data, and environmental data.

[1406] (Claim 3)

[1407] 2. The system of claim 1, wherein the future prediction means includes means for predicting future events and generating different future scenarios using machine learning, deep learning, and statistical modeling techniques.

[1408] "Application Example 1"

[1409] (Claim 1)

[1410] data collection means;

[1411] A method for predicting the future using machine learning models,

[1412] means for generating a virtual reality space based on the prediction data;

[1413] A means of recreating future events in a virtual reality space;

[1414] A means to recreate future driving scenarios for autonomous vehicles in virtual reality based on traffic data; and

[1415] A system including:

[1416] (Claim 2)

[1417] 10. The system of claim 1, wherein the data collection means comprises means for collecting data from a plurality of data sources including weather data, economic indicators, social statistics, technology trend data, environmental data, and traffic data.

[1418] (Claim 3)

[1419] 2. The system of claim 1, wherein the future prediction means includes means for predicting future events and generating different future scenarios using machine learning, deep learning, and statistical modeling techniques.

[1420] "Example 2: Combining Emotion Engines"

[1421] (Claim 1)

[1422] data collection means;

[1423] A method for predicting the future using machine learning models,

[1424] means for generating a virtual reality space based on the prediction data;

[1425] A means of recreating future events in a virtual reality space;

[1426] means for recognizing a user's emotion;

[1427] a means for dynamically adjusting a scenario or environment within the virtual reality space based on the emotion;

[1428] A system including:

[1429] (Claim 2)

[1430] 10. The system of claim 1, wherein the data collection means comprises means for collecting data from a plurality of data sources including meteorological data, economic indicators, social statistics, technology trend data, and environmental data.

[1431] (Claim 3)

[1432] 2. The system of claim 1, wherein the future prediction means includes means for predicting future events and generating different future scenarios using machine learning, deep learning, and statistical modeling techniques.

[1433] "Application example 2 when combining emotion engines"

[1434] (Claim 1)

[1435] data collection means;

[1436] A method for predicting the future using machine learning models,

[1437] means for generating a virtual reality space based on the prediction data;

[1438] A means of recreating future events in a virtual reality space;

[1439] An emotion recognition means;

[1440] means for dynamically adjusting a scenario or environment within the virtual reality space based on the emotion data;

[1441] A system including:

[1442] (Claim 2)

[1443] 10. The system of claim 1, wherein the data collection means comprises means for collecting data from a plurality of data sources including weather data, economic indicators, social statistics, technology trend data, environmental data, and consumer behavior data.

[1444] (Claim 3)

[1445] 2. The system of claim 1, wherein the future prediction means includes means for predicting future events and consumer behavior using machine learning, deep learning, and statistical modeling techniques and generating different future scenarios. [Explanation of symbols]

[1446] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. data collection means; A method for predicting the future using machine learning models, means for generating a virtual reality space based on the prediction data; A means of recreating future events in a virtual reality space; A system including:

2. 2. The system of claim 1, wherein the data collection means includes means for collecting data from a plurality of data sources including meteorological data, economic indicators, social statistics, technology trend data, and environmental data.

3. The system of claim 1 , wherein the future prediction means includes means for predicting future events using machine learning, deep learning, and statistical modeling techniques and generating different future scenarios.

4. 2. The system according to claim 1, wherein the means for generating a virtual reality space includes means for generating a high-quality virtual space incorporating visual, auditory, and tactile elements.

5. The system of claim 1, wherein the means for recreating future events in a virtual reality space includes a means for the user to specify a specific date or conditions to experience future events and a means for the user to interact with the generated future characters.

Images