Information processing device, method, program

Abstract

This invention provides an information processing device, method, and program that realize realistic simulations in a virtual space that replicates the real world. [Solution] The method includes setting up a model that outputs a response in accordance with data related to the nerves of a living organism in an object representing a living organism; defining a virtual space that reproduces a real space and placing an object representing a living organism in that virtual space; identifying input data that affects the object based on the data in the virtual space; generating control data for the object based on output data obtained by inputting the identified input data into the model; and operating the object in the virtual space based on the generated control data.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, method, and program.

Background Art

[0002] In recent years, digital twin technology that faithfully reproduces physical real-world spaces (cities, buildings, infrastructure, etc.) in a virtual space using sensing technology and 3D modeling technology has attracted attention. This technology performs analysis, future prediction, and optimization of events in the real world by executing various simulations in the virtual space, and is being applied in a wide range of fields such as urban development, traffic planning, and disaster countermeasures.

[0003] One of the main application examples of digital twins is pedestrian flow simulation that simulates human behavior in a virtual space. Conventionally, multi-agent simulation (MAS) methods have been widely used for this type of pedestrian flow simulation. In general MAS, predefined behavior rules such as "move along the shortest path to the destination" and "avoid collisions with obstacles and other people (e.g., social force model etc.)" are set for agents (objects) that simulate individual humans, and the agents are operated in accordance with the rules.

[0004] Patent Document 1, described below, addresses the problem of "predicting pedestrian flow when a new building is constructed in a target area, taking into account the attractive effect of the new building," and states that "the pedestrian flow prediction device 14 acquires a trained model that, upon input of building data representing the building and building surrounding data representing the building's surrounding environment, outputs an attraction score representing the degree to which the building attracts people. The pedestrian flow prediction device 14 acquires the attraction score of a virtual building by inputting building data and building surrounding data of a virtual building to be constructed in the target area after development into the trained model. Based on the attraction score, the pedestrian flow prediction device 14 determines the transition probability to each location of an agent representing a person when executing a multi-agent simulation, and executes the multi-agent simulation based on the transition probability." [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2024-110651 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, since human behavior in the real world does not always follow only simple, predefined rules, conventional rule-based simulations have not been able to adequately reproduce the complex behavioral principles of real humans.

[0007] This disclosure is made in view of the aforementioned problems with the prior art, and aims to provide a technology that enables realistic simulations in a virtual space that reproduces the real world. [Means for solving the problem]

[0008] According to one embodiment shown in this disclosure, a program is provided that is executed by a computer having one or more computer processors. The memory unit is configured to store data of a virtual space that reproduces a real space and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response in accordance with data related to the nerves of the living organisms, and the program causes one or more computer processors to execute the following steps: define a virtual space that reproduces a real space; place objects representing living organisms in the virtual space; identify input data that affects the objects representing living organisms based on the data set up in the virtual space; input the input data identified in the identification step into the model set up on the object and generate control data for the objects representing living organisms based on the output data output from the model; and operate the objects representing living organisms in the virtual space based on the control data generated in the generation step. [Effects of the Invention]

[0009] According to this disclosure, it is possible to realize realistic simulations in a virtual space that reproduces the real world. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows the configuration of System 1. [Figure 2] Figure 2 shows the configuration of server 20. [Figure 3] Figure 3 shows the configuration of terminal 10. [Figure 4] Figure 4 illustrates the concept of an integrated digital twin, which combines a digital twin of space with a digital twin of a living organism. [Figure 5] Figure 5 is a conceptual diagram of the internal structure of the model set for a living organism object. [Figure 6] Figure 6 shows the data structure of the virtual space database 211. [Figure 7] Figure 7 shows the data structure of the living organism object database 212. [Figure 8] Figure 8 shows the data structure of the scenario database 213. [Figure 9] Figure 9 shows the overall processing flow of the simulation. [Figure 10] Figure 10 shows the process flow for determining the autonomous actions of individual objects. [Figure 11] Figure 11 shows the process flow for identifying input data for an object. [Figure 12] Figure 12 shows the flow of processes in urban development and spatial design. [Figure 13] Figure 13 is an example of an operation screen that provides an overview of how numerous objects move and act autonomously in a virtual space. [Figure 14] Figure 14 shows an example of an operation screen that displays the viewpoint of a specific object and detailed information about that object. [Figure 15] Figure 15 is an example of an operation screen that shows how multiple objects communicate with each other (through chat and voice communication), how information spreads, and how their behavior changes. [Figure 16] Figure 16 is an example of an operation screen showing how objects take evacuation actions based on their perception in a disaster scenario where routes are blocked. [Figure 17] Figure 17 shows an example of an operation screen that displays simulation results visualizing pedestrian flow, congestion, and psychological state in a specific layout plan. [Figure 18] Figure 18 shows an example of an operation screen that displays a comparison of visual changes in the same virtual space under different time periods, seasons, and weather conditions. [Figure 19] Figure 19 shows an example of a user interface (UI) where the user inputs requests in natural language, and an operation screen displaying recommended design proposals generated by AI based on those inputs. Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also identical. Therefore, detailed descriptions thereof will not be repeated.

[0012] <Outline of Embodiment> Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, the outline of the invention according to the present embodiment will be described, and then specific configurations and operations will be explained.

[0013] The present embodiment relates to digital twin technology that reproduces the real world in a virtual space, and particularly relates to an "integrated digital twin system" that simulates the behaviors of living organisms (mainly humans) acting in the virtual space in a form that more closely matches reality.

[0014] In conventional human behavior simulation (for example, crowd flow simulation), the top-down approach that assigns predefined simple behavior rules (e.g., shortest movement to a destination, collision avoidance, etc.) to objects (agents) that simulate individual humans has been the mainstream.

[0015] However, since actual human behavior is autonomously determined based on complex factors such as perception of the surrounding environment, personal background (personality and experience), and internal state at each moment (emotion and cognition), conventional technologies have had limits in reproducibility and prediction accuracy.

[0016] Humans perceive diverse information from their surroundings through their five senses (visual information, weather conditions, the words and actions of others, etc.), and autonomously make decisions through complex thought processes based on their individual background information (personality, experience, social attributes, etc.) and their internal state at any given time (emotions, cognition, physiological needs, etc.). For example, even in the same physical environment, one person might choose a detour due to discomfort from congestion, while another might head towards the center of the crowd out of curiosity. Furthermore, in unexpected situations such as disasters, anxiety and the spread of information from others have a significant impact on behavior.

[0017] Conventional technologies have struggled to simulate such individual cognitive processes and autonomous decision-making mechanisms, limiting their ability to realistically reproduce and predict complex social phenomena and crowd behaviors that emerge from the interactions of many people.

[0018] To solve this problem, the integrated digital twin system according to this embodiment integrates a "spatial digital twin" that simulates the physical environment and a "human digital twin" that simulates human cognitive processes, and provides a simulation environment in which the two interact with each other.

[0019] Specifically, this system defines a virtual space that replicates terrain, environment, and physical laws. Within this virtual space, objects representing living organisms are placed, each equipped with a "cognitive model" (for example, a Large-Scale Language Model (LLM) can be used) that simulates the neural activity and thought processes of living organisms.

[0020] During simulation execution, the system identifies the input data that each object perceives based on the state of the virtual space (e.g., environmental changes, the words and actions of others). Next, this input data is fed into the object's cognitive model. The cognitive model interprets the situation based on the input data and the object's unique background information and internal state, and outputs data related to cognition and emotion. Based on this output data, the system generates control data that determines the object's next action (movement, speech, etc.), and makes the object operate autonomously in the virtual space.

[0021] In this way, in a virtual space that replicates real space, a model is set up for an object representing a living organism that outputs a response in accordance with data related to the organism's nervous system (a model that outputs a response in accordance with data related to the organism's cognitive or thought processes). Then, input data identified based on the state of the virtual space (changes in the environment, the actions of others, etc.) is input into the model, and the object's behavior is autonomously controlled based on its output (cognition, emotions, etc.).

[0022] This enables a bottom-up simulation in which individual objects perceive and judge their surroundings and act accordingly. As a result, it becomes possible to create more realistic behavioral simulations that reflect the diversity and complex psychological processes of humans, which were difficult to reproduce with conventional top-down approaches. This contributes to more accurate predictions and the acquisition of new insights in areas such as urban planning evaluation, disaster evacuation behavior prediction, and analysis of complex social phenomena.

[0023] As described above, this technology simulates autonomous behavior based on the neural activity and cognition of individual living organisms (humans), enabling a "bottom-up" simulation where individual objects perceive and judge their surroundings and act accordingly, and the resulting interactions create an overall phenomenon. This makes it possible to perform more accurate and realistic behavioral simulations that reflect the diversity and complex psychological processes of humans, which were difficult to reproduce with conventional technologies. This technology is expected to have applications in a wide range of fields, such as evaluating spatial design in urban development, predicting crowd behavior during disasters, and analyzing complex social phenomena. In this embodiment, the "virtual space" encompasses both a reproduction of physical space (physical space) and an "information space" that reproduces information networks such as SNS, and it is possible to simulate by linking these two. For example, it is possible to simulate objects moving in physical space while simultaneously exchanging information in the information space. Complex interactions can be reproduced, such as events in physical space (e.g., witnessing a disaster) triggering transmission in the information space (e.g., posting on SNS), which then influences the cognition of other objects, leading to changes in behavior in physical space (e.g., initiating evacuation).

[0024] The specific configuration and processing of this system will be explained below using diagrams.

[0025] <1.1 System Configuration Diagram> Figure 1 shows the configuration of System 1.

[0026] System 1, shown in Figure 1, includes a server 20, user terminals 10 and 10A, an artificial intelligence (large-scale language model) service server 95 (hereinafter sometimes referred to as "large-scale language model service server 95"), and a device control system 98. These devices are connected via a network 80.

[0027] Server 20 provides a service for building a simulation environment using a digital twin.

[0028] The device control system 98 is a system that controls various devices placed in the real world. By communicating with the server 20, the device control system 98 receives control signals transmitted from the server 20 and controls the various devices by interpreting the content of the control signals.

[0029] Terminal 10 is a terminal used by a user of the services provided by Server 20. In the illustrated example, terminals 10, terminal 10A, etc., are shown as terminals used by users of the services provided by Server 20, but each user operates their own terminal.

[0030] Server 95 of the Large-Scale Language Model Service is a server that executes language processing tasks using language models built through learning processes including artificial intelligence (AI). An LLM (Large Language Model) is one that has been pre-trained on a large amount of data (such as text data), such as web content on the internet or data stored in a designated database, or has been pre-trained on a large amount of data stored in a designated database, and can execute various language processing tasks by being given a task.

[0031] The server 95 of the large-scale language model service accepts prompt input in the form of text, images, audio, etc., and generates and responds with answers to those prompts. Examples of LLMs include GPT-4 and GPT-4o developed by OpenAI, and Gemini developed by Google.

[0032] The configuration of each device is described below.

[0033] The server 20 includes a communication interface 22, an input / output interface 23, memory 25, storage 26, and a processor 29.

[0034] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0035] Input / Output IF23 functions as an interface between an input device for receiving user input operations and an output device for presenting information to the user.

[0036] Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0037] Storage 26 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0038] The processor 29 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0039] Terminal 10 can be implemented, for example, as follows: • Handheld mobile devices such as smartphones and tablets • Stationary PC (Personal Computer), Laptop PC • Wearable devices worn by the user (watch-type, glasses-type, etc.) Terminal 10 includes a communication interface (IF) 12, an input device 13, an output device 14, memory 15, storage 16, and a processor 19.

[0040] The communication interface 12 is an interface for inputting and outputting signals so that terminal 10 can communicate with an external device.

[0041] The input device 13 is a device for receiving input operations from the user (for example, a touch panel, touchpad, pointing device such as a mouse, keyboard, etc.).

[0042] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0043] Memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0044] Storage 16 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0045] The processor 19 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0046] <1.2 Functional Configuration of Server 20> Figure 2 shows the configuration of server 20. As shown in Figure 2, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0047] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0048] The memory unit 202 stores various databases, such as the virtual space database 211, the living organism object database 212, the scenario database 213, the model data 214, and the simulation results 215.

[0049] The virtual space database 211 stores data for defining the virtual space (such as terrain data, environmental model data, physical law data, and layout plan data).

[0050] The organism object database 212 stores information about organism objects (background information, history of internal state, etc.).

[0051] Scenario database 213 stores data that defines the conditions for the simulation (such as disaster scenarios).

[0052] Model data 214 is an area that stores the cognitive model's program and trained parameters (such as LLM parameters). It can store parameters for specialized models that have been fine-tuned using datasets related to specific domains (e.g., past purchase history or behavioral survey data of specific social groups). Furthermore, different performance levels and types of cognitive models can be used or mixed depending on the simulation objective and required accuracy. For example, to grasp the overall trends of a large crowd, a lightweight LLM with low computational cost or a simpler decision-making model other than an LLM (e.g., rule-based inference engine, reinforcement learning model) can be applied to numerous objects. On the other hand, for objects with specific important roles (e.g., evacuation guides, influencers) or objects requiring detailed analysis, a high-performance LLM (e.g., GPT) can be applied to simulate complex situational judgments and influence on others in detail. The control unit 203 selects and applies the optimal model (model ID) based on the background information and role of each object.

[0053] Simulation Results 215 is the area that stores the results (logs, analysis data) of the executed simulation.

[0054] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs the functions shown below: virtual space definition module 2033, object placement module 2034, simulation execution control module 2035, input data identification module 2036, model processing module 2037, control data generation module 2038, operation execution module 2039, spatial state update module 2040, scenario control module 2041, recommended value generation module 2042, and result output control module 2043.

[0055] The virtual space definition module 2033 is a program module that reads various data from the memory unit and constructs a virtual space that serves as the basis for the simulation. This module is also responsible for reproducing natural phenomena and defining the layout plan of buildings and other structures. The reproduction of natural phenomena is achieved, for example, using various physics engines (e.g., rigid body physics engine, fluid simulation engine). Based on the physical law data and terrain data defined in the virtual space database 211, it calculates collision detection between objects, the motion of objects, and the behavior of fluids according to weather conditions (e.g., river flooding, analysis of wind between buildings).

[0056] The object placement module 2034 is a program module that reads various information from the memory unit, creates and places objects in the virtual space, and sets their background information and initial state.

[0057] The Simulation Execution Control Module 2035 is a program module that manages the progress of the entire simulation and executes the simulation in cooperation with the modules described below.

[0058] The Input Data Identification Module 2036 is a program module that identifies the input data perceived by each object (visual information, simulated biometric information, information received from other objects, etc.) based on the current state of the virtual space.

[0059] The model processing module 2037 is a program module that inputs the input data received from the input data identification module 2036 into the cognitive model (e.g., LLM) of each object, and generates output data related to cognition and emotion.

[0060] The control data generation module 2038 is a program module that determines the next action to be taken and generates control data based on the data output by the model processing module 2037 and the background information of the object.

[0061] The execution module 2039 is a program module that operates objects in a virtual space based on the control data generated by the control data generation module 2038.

[0062] The spatial state update module 2040 is a program module that updates the virtual space and the internal state of objects, reflecting the effects of object movements and environmental changes over time.

[0063] The Recommendation Value Generation Module 2042 is a program module that uses AI to automatically generate recommendations for the arrangement and design of a virtual space based on natural language requests and target state settings from the user. This AI, for example, uses a generative model (Generative AI) trained on numerous architectural designs and their evaluation data to generate new design proposals that match the input requests. Alternatively, it can use an optimization algorithm to search for an arrangement that maximizes the psychological state (e.g., "comfort level") calculated by this simulation, using the user's requests as constraints. The result output control module 2043 is a program module that stores simulation results in a memory unit and outputs signals to control real-world equipment and systems as needed. For example, if the results of a pedestrian flow simulation in the virtual space predict excessive congestion in a specific area in the future, it is possible to control equipment corresponding to that area in the real world. Specifically, it outputs control signals to the equipment control system 98, such as changing the display content of digital signage (display devices) to guide people to alternative routes, or adjusting the control parameters of traffic signals. This makes it possible to prevent or mitigate problems in the real world based on future predictions in the virtual space.

[0064] <1.3 Configuration of Terminal 10> Figure 3 shows the configuration of terminal 10.

[0065] As shown in Figure 3, terminal 10 includes multiple antennas (antenna 111, antenna 112), communication units corresponding to each antenna (first communication unit 120, second communication unit 121), an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a storage unit 180, and a control unit 190. Terminal 10 also has functions and configurations not specifically shown in Figure 3 (for example, a battery for maintaining power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in Figure 3, each block included in terminal 10 is electrically connected by a bus or the like.

[0066] Antenna 111 radiates signals emitted by terminal 10 as radio waves. Antenna 111 also receives radio waves from space and provides the received signals to first communication unit 120.

[0067] Antenna 112 radiates signals emitted by terminal 10 as radio waves. Antenna 112 also receives radio waves from space and provides the received signals to the second communication unit 121.

[0068] The first communication unit 120 performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 111 in order to communicate with other wireless devices. The second communication unit 121 also performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 112 in order to communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are a communication module that includes a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first communication unit 120 and the second communication unit 121 perform modulation and demodulation, frequency conversion, etc., of the wireless signals transmitted and received by the terminal 10, and provide the received signal to the control unit 190.

[0069] The input device 130 has a mechanism for receiving user input operations. Specifically, the input device 130 is configured as a touchscreen and includes a touch-sensitive device 131. The touch-sensitive device 131 receives user input operations of the terminal 10. The touch-sensitive device 131 detects the user's contact position with the touch panel, for example, by using a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the user's contact position detected by the touch panel to the control unit 190 as an input operation.

[0070] The display 132 displays data such as images, videos, and text in accordance with the control of the control unit 190. The display 132 is implemented by, for example, an LCD or an organic EL display.

[0071] The audio processing unit 140 modulates and demodulates the audio signal. The audio processing unit 140 modulates the signal received from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is implemented, for example, by an audio processing processor. The microphone 141 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 140. The speaker 142 converts the audio signal received from the audio processing unit 140 into sound and outputs the sound to the outside of the terminal 10.

[0072] The location information sensor 150 is a sensor that detects the location of the terminal 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current location of the terminal 10, which is equipped with a GPS module, is detected based on the received signals.

[0073] Camera 160 is a device that receives light using a photodetector and outputs it as an image. Camera 160 is, for example, a depth camera that can detect the distance from camera 160 to the object being photographed.

[0074] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10.

[0075] The storage unit 180 is composed of, for example, flash memory and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.

[0076] The control unit 190 controls the operation of the terminal 10 by reading the program stored in the memory unit 180 and executing the instructions contained in the program. The control unit 190 is, for example, an application processor. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a memory control unit 195.

[0077] The operation reception unit 191 processes input operations from the user to an input device such as a touch-sensitive device 131. Based on the coordinate information of the touch-sensitive device 131 where the user's finger or the like has made contact, the operation reception unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation.

[0078] The transmitting / receiving unit 192 performs processing to enable the terminal 10 to send and receive data with an external device such as a server 20 in accordance with a communication protocol.

[0079] The data processing unit 193 performs calculations on the data received as input by the terminal 10 according to the program and outputs the calculation results to memory or other locations.

[0080] The notification control unit 194 performs the following processes: displaying the display image on the display 132, outputting sound to the speaker 142, and generating vibrations.

[0081] The memory control unit 195 controls the storage of data to the memory unit 180.

[0082] The various types of information stored by the memory unit 180 will now be explained. In a given scenario, the memory unit 180 stores various types of information, such as user information 181.

[0083] User information 181 is information about a user who uses the services of server 20.

[0084] <1.4 Explanation of Concepts> Figure 4 illustrates the concept of an integrated digital twin, which combines a digital twin of space with a digital twin of a living organism.

[0085] As illustrated, in a spatial digital twin, simulations are performed through the interaction of data on terrain and structures, models of the environment (weather, time) that influences those terrain and structures, and models of physical laws. For example, it is possible to simulate whether or not a river will overflow due to rising water levels caused by rainfall, and to calculate the flood situation if a river does overflow.

[0086] Furthermore, in a digital twin of a living organism (a human in the illustrated example), information about the person's sociality and background, a cognitive model (emotions, thoughts, etc.), and a biological / physical model interact with each other to output the biological responses that occur in the person in response to the circumstances surrounding the person (crowding, weather conditions such as temperature, occurrence of disasters, etc.), cognition based on the information the person perceives, and the resulting emotions and behaviors of the person.

[0087] In this way, the simulation is performed by generating perceptions in an agent model corresponding to a human based on the state of the space, and then reflecting the agent model's actions in relation to the state of the space.

[0088] Figure 5 is a conceptual diagram of the internal structure of the model set for a living organism object.

[0089] As illustrated, the agent model references background information (personality, history, social attributes, income, long-term memory, values, etc.) and internal state (current situation; emotions (pleasure, displeasure, etc.), physiological needs (hunger, etc.), short-term cognition, mood, etc.). Furthermore, information perceived by the organism (visual information (visual information determined based on a virtual camera, etc.), environmental information (weather, physical phenomena, etc.), the actions of other agents (group behavior, etc.)) influences the internal state and is input into the cognitive model, where thinking and judgment are made based on this information. For example, within the cognitive model, a language processing system such as a Large-Scale Language Model (LLM) can interpret the situation and output decision-making and action plans. The output of the cognitive model then triggers actions (movement, movement, etc.) and communication (speech, communication, etc.) of the organism object.

[0090] <2 Data Structure> Figure 6 shows the data structure of the virtual space database 211. The virtual space database 211 includes the following items: "Space ID", "Space Name", "Reference Date and Time", "Coordinate System Definition", "Data Source Information", "Level of Detail (LOD)", "Terrain Model File Path", "Feature Semantic Information File Path", "Time Progression Rate", "Weather", "Temperature (°C)", "Humidity (%)", "Wind Speed ​​(meters per second)", "Season", "Solar Altitude Angle", "Physics Engine Configuration File", "Gravitational Acceleration", "Standard Air Resistance Coefficient", "Structure ID", "Structure Model File Path", "Placement Location (Latitude, Longitude, Altitude)", "Usage Type", "Road Network File Path", and "Dynamic Passability Information".

[0091] The item "Spatial ID" is an ID used to uniquely identify a virtual space.

[0092] The item "Space Name" is information indicating the name of the virtual space.

[0093] The item "Reference Date and Time" is the date and time information that serves as the basis for the simulation.

[0094] The item "Coordinate System Definition" contains the definition information for the geographic coordinate system to be used.

[0095] The "Data Source Information" field indicates the source of the topographic data used.

[0096] The "Level of Detail (LOD)" field is attribute information that indicates the level of detail of terrain and structures.

[0097] The item "Terrain Model File Path" is path information indicating the location where the 3D terrain model file is saved.

[0098] The item "Feature Semantic Information File Path" is path information indicating the location where the semantic information files for terrain and features (e.g., CityGML) are stored.

[0099] The item "Time Progression Rate" indicates the rate at which simulated time progresses relative to real time.

[0100] The "Weather" item is attribute information that indicates the current weather (which can change over time).

[0101] The item "Temperature (°C)" shows the current temperature.

[0102] The item "Humidity (percent)" shows the current humidity.

[0103] The item "Wind speed (meters per second)" shows the current wind speed.

[0104] The "Season" attribute is attribute information that indicates the current season (which affects visual representation and vegetation).

[0105] The item "Solar Altitude Angle" indicates the current position of the sun (used in calculating sunlight and shadows).

[0106] The item "Physics Engine Configuration File" is the path to the configuration file for the physics engine to be used.

[0107] The item "Gravitational Acceleration" is information that indicates the gravitational acceleration (m / s^2) applied to the virtual space.

[0108] The item "Standard drag coefficient" indicates the default drag coefficient.

[0109] The "Structure ID" field is an ID used to uniquely identify the structure (building, etc.) to be placed.

[0110] The item "Structural Model File Path" is path information indicating the location where the 3D model (BIM / CAD, etc.) file of the structure is saved.

[0111] The item "Location (Latitude, Longitude, Altitude)" contains information indicating the location of the structure.

[0112] The "Usage Type" field is attribute information that indicates the intended use of the structure.

[0113] The item "Road Network File Path" is path information indicating the location where the road network (nodes and links) definition files are stored.

[0114] The item "Dynamic Passability Information" is status information that indicates the dynamic passability status based on the scenario.

[0115] Figure 7 shows the data structure of the living organism object database 212. The living organism object database 212 includes the following fields: "Object ID", "Object Name", "Activity Status", "Model ID", "Current Location (X-coordinate)", "Current Location (Y-coordinate)", "Current Location (Z-coordinate)", "Orientation (Angle)", "Movement Speed ​​(meters per second)", "Personality Trait (Extroversion)", "Personality Trait (Conscientiousness)", and "Background / Work History". This includes the following items: "Social Attributes (Occupation)", "Social Attributes (Household Composition)", "Income Level", "Long-Term Memory Data", "Current Cognitive Content", "Emotional Parameter (Joy)", "Emotional Parameter (Anger)", "Emotional Parameter (Irritability)", "Physiological Needs (Hunger)", "Simulated Heart Rate (Beats Per Minute)", "Simulated Body Temperature (°C)", and "Simulated Sweat Volume".

[0116] The "Object ID" field is an ID used to uniquely identify an object (agent).

[0117] The "Object Name" field contains information indicating the name (display name) of the object.

[0118] The "Activity Status" field is status information that indicates the activity status of an object (active, stopped, etc.).

[0119] The "Model ID" field is an ID used to identify the cognitive model being applied.

[0120] The item "Current Location (X-coordinate)" is information indicating the current X-coordinate in the virtual space.

[0121] The item "Current Location (Y-coordinate)" is information indicating the current Y-coordinate in the virtual space.

[0122] The item "Current Location (Z-coordinate)" is information indicating the current Z-coordinate in the virtual space.

[0123] The "Orientation (Angle)" field indicates the direction (angle) the object is facing.

[0124] The item "Movement Speed ​​(meters per second)" indicates the current movement speed.

[0125] The item "Personality Trait (Extraversion)" is attribute information that indicates one of the personality traits (e.g., the Big Five personality indicator).

[0126] The item "Personality Trait (Conscientiousness)" is attribute information that indicates one of the personality traits.

[0127] The "Career / Work History" field contains information about the object's career and work history.

[0128] The item "Social Attributes (Occupation)" is attribute information that indicates social attributes (occupation).

[0129] The item "Social Attributes (Household Composition)" is attribute information that indicates social attributes (household composition).

[0130] The item "Income Level" is information related to income (which influences behavioral principles).

[0131] The item "Long-term memory data" refers to data related to past experiences and knowledge (vector data, etc. For example, data that represents past experiences and knowledge as numerical vectors that retain their meaning). This long-term memory data is, for example, data that has been vectorized by an embedding model from a summary text of past experiences. This is used to enable judgments that are in line with an individual's experience by retrieving memories related to the current situation using RAG (Retrieval-Augmented Generation) technology (for example, vector similarity search) and including the search results as input (prompt) to a cognitive model (LLM).

[0132] The item "Current Cognitive Content" is information that indicates the current situation and thoughts.

[0133] The item "Emotional Parameter (Joy)" is evaluation result information indicating the intensity of the emotion (e.g., 0.0 to 1.0).

[0134] The item "Emotional Parameter (Anger)" is evaluation result information indicating the intensity of emotion.

[0135] The item "Emotional Parameter (Anxiety)" is evaluation result information indicating the intensity of emotion.

[0136] The item "Physiological Needs (Hunger Level)" is evaluation result information indicating the intensity of physiological needs (e.g., 0.0 to 1.0).

[0137] The item "Simulated Heart Rate (beats per minute)" shows information indicating the heart rate in the simulation.

[0138] The item "Simulated Body Temperature (°C)" is information that shows the body temperature in the simulation.

[0139] The item "Simulated Sweat Volume" indicates the level of sweat volume in the simulation.

[0140] Figure 8 shows the data structure of the scenario database 213. The scenario data database 213 includes the following items: "Scenario ID", "Scenario Name", "Target Space ID", "Event ID", "Event Type", "Occurrence Start Time (Absolute Time)", "Occurrence Start Time (Relative Time)", "Occurrence Location (Coordinates)", "Event Scale Parameter", "Impact ID", "Impact Type", "Target Link ID", "Passability Status", "Impact Start Time (Relative Time)", "Environmental Change Parameter", "Affected Object Conditions", "Notification Content to Object", and "Internal State Change Content".

[0141] The "Scenario ID" field is an ID used to uniquely identify a scenario.

[0142] The item "Scenario Name" contains information indicating the name of the scenario.

[0143] The item "Target Space ID" is an ID used to identify the virtual space to which the scenario is applied.

[0144] The "Event ID" field is an ID used to identify individual events that occur within a scenario.

[0145] The "Event Type" field is attribute information that indicates the type of event that will occur (disaster, accident, event, etc.).

[0146] The item "Start Time of Occurrence (Absolute Time)" indicates the absolute time when the event occurred.

[0147] The item "Start Time (Relative Time)" indicates the relative time (in seconds) from the start of the simulation.

[0148] The item "Location (Coordinates)" indicates the central coordinates where the event occurs.

[0149] The item "Event Scale Parameter" is a parameter that indicates the scale of an event (e.g., seismic intensity, fire scale).

[0150] The "Impact ID" field is an ID used to identify the specific impact of an event.

[0151] The item "Impact Type" is attribute information that indicates the type of impact.

[0152] The item "Target Link ID" is an ID used to identify the road or pathway link that will be blocked.

[0153] The item "Accessibility Status" is status information indicating the accessibility status of the target link.

[0154] The item "Influence Start Time (Relative Time)" indicates the relative time (in seconds) when the influence begins to occur.

[0155] The item "Environmental Change Parameters" contains information that shows changes in environmental parameters due to an event (e.g., reduced visibility due to smoke).

[0156] The item "Affected Object Conditions" is attribute information that indicates the conditions for the objects that are affected.

[0157] The item "Notification content to object" specifies the content of the information to be notified (recognized) by the target object.

[0158] The item "Internal State Change Details" contains information that shows the changes in the object's internal state (emotions / goals).

[0159] Furthermore, although not shown in the diagram, in addition to events occurring over time, events may also be dynamically triggered by "specific states in the simulation," and it may be possible to enable scenario branching depending on the situation, such as "sending a notification to surrounding objects prompting them to take a detour when the congestion level in a specific area exceeds a threshold." For example, the scenario database 213 may have the following subfields, and the simulation may be started in response to the occurrence of a specific state.

[0160] Item 'Trigger Conditions': This is information that defines the dynamic conditions under which an event occurs. In addition to time (absolute / relative), it may be possible to set conditions based on the state of the simulation, such as 'Population density in a specific area > 10 people / m^2' or 'Emotion parameter (anger) of object A > 0.9'.

[0161] <3 operations> Figure 9 shows the overall processing flow of the simulation.

[0162] In step S911, terminal 10 accepts the user's settings for running the simulation (such as the virtual space data to be used, the scenario, and the types and number of objects to be placed), and sends a command to start the simulation to server 20.

[0163] In step S921, the server 20 receives a start command from the terminal 10 and begins the initial setup of the simulation.

[0164] In step S922, the server 20 reads the specified virtual space data (terrain, environmental model, physical laws, layout plan, etc.) from the memory unit and constructs the virtual space on the simulation environment. At this time, or during simulation execution (S927), the server 20 can also collaborate with external IoT (Internet of Things) platforms and real-time data distribution services to acquire the latest sensor data from the real world (e.g., traffic volume, pedestrian flow data, weather information, infrastructure operating status, etc.) and dynamically reflect it in the state of the virtual space (each parameter of the virtual space database 211). This enables real-time simulations that instantly incorporate events occurring in the real world into the virtual space and predict current or near-future situations with greater accuracy.

[0165] In step S923, the server 20 reads object information (model data, background information, etc.) from the memory unit, generates and places a living organism object (AI agent) at its initial position in the virtual space, and sets its initial state.

[0166] (Simulation loop) In step S924, the server 20 identifies the input data (visual information, environmental information, etc.) perceived by each object based on the current state of the virtual space (time, environment, position and behavior of other objects, etc.). This simulation loop can be executed synchronously at predetermined time intervals (time steps). That is, after completing the processing in S924-S926 for all objects in that time step, the state is updated collectively in S927 and the process proceeds to the next time step. Alternatively, it is possible to implement an asynchronous type in which each object independently executes processing asynchronously. Furthermore, the processing in S924-S926, especially the computationally intensive S925 (control data generation by the model), can be accelerated by performing parallel distributed processing using multiple computers (servers) or multiple processor cores. For example, techniques such as batch processing of inference requests to the cognitive model (LLM) can be used.

[0167] In step S925, the server 20 inputs the identified input data into the model of each object (e.g., LLM) and causes the model to generate control data for determining the next action.

[0168] In step S926, the server 20 executes the actions (movement, speech, etc.) of each object in the virtual space based on the generated control data.

[0169] In step S927, the server 20 advances the simulation time and updates the virtual space and the internal state of the objects, reflecting the effects of the objects' actions. For example, if the physics engine calculates the spread of a fire, the environmental information of the fire area (temperature rise, smoke density) is updated. This updated environmental information is identified as input data for objects present in that area in the next loop (S924), influencing the objects' perception and actions (e.g., sensing danger and starting to evacuate). Conversely, the system can also be configured so that the actions of objects (e.g., opening and closing doors) affect the parameters of the physics simulation (e.g., airflow, fire spread rate).

[0170] In this update process, server 20 aggregates the results of all object actions during the time step (e.g., changes in position coordinates). Based on these aggregated results, it calculates new environmental parameters (e.g., population density in a specific area, road congestion, etc.) and updates the information in the virtual space database 211. As a result, in the next time step, each object will be able to perceive the environment after it has been changed by the actions of other objects as new input data.

[0171] In step S928, the server 20 determines whether the simulation has met a predetermined termination condition (e.g., the end of a specified period), and if not, returns to the process in step S924. Furthermore, as part of this state update, it is also possible to process the experiences of each object and reflect them in long-term memory. For example, each time a day passes in the simulation time, the day's activity log and cognitive changes are input into the object's own cognitive model (LLM) and summarized to generate summary text of experiences such as "I liked the new cafe" or "I was scared by the fire alarm." This summary text is then vectorized and added to the object's long-term memory database 212. This allows the object to learn through its experiences in the simulation, and its future behavior can change to be more complex and realistic.

[0172] In step S929, if the termination conditions are met, the server 20 saves the simulation results (action log, state change log, etc.) to the storage unit (termination process).

[0173] In step S912, terminal 10 receives the simulation results from server 20 and presents them to the user using a 3D viewer, graphs, etc.

[0174] Figure 10 shows the process flow for determining the autonomous actions of individual objects.

[0175] In step S1022, the server 20 obtains the input data perceived by the target object.

[0176] In step S1023, the server 20 reads background information (personality, history, social attributes, income, etc.) and the current internal state (emotions, physiological needs, etc.) of the target object from the memory unit.

[0177] In step S1024, the server 20 inputs the acquired input data, background information, and internal state into the cognitive model (e.g., LLM) set up for the target object. Specifically, this diverse information can be formatted as a natural language prompt like the following and given to the LLM: "You are a company employee in your 20s with an extroverted personality. Your current emotion is 'anxiety,' and you are feeling 'hungry' as a physiological need. You currently see a 'crowded intersection,' and your surroundings are 'noisy.' You have heard from a nearby colleague that 'a cafe in front of the station has reopened after renovations.' Based on these circumstances, please describe your next thought and action plan." (As a thought and action plan, for example, "Cross the pedestrian crossing to head to the cafe" might be output.) Specifically, for example, if the input data is "Visual information: A large number of people are gathered in front," the internal state is "Anxiety level 0.8," and the background information is "Personality: Extroverted," then Server 20 integrates these and generates a natural language prompt such as, "You are an extrovert. You are currently feeling anxious (level 0.8). There is a crowd of people in front of you. Interpret this situation and plan your next course of action," and inputs it into the LLM.

[0178] In step S1025, the server 20 instructs the cognitive model to interpret the situation based on the input information and generate output data of the cognition and emotion of the target object. At this time, by instructing the LLM to output in a specific format (e.g., JSON format), the cognitive content (text) and emotion parameters (numerical values) can be obtained as structured data. Alternatively, the natural language text output by the LLM can be parsed and converted into a predetermined data structure.

[0179] In step S1026, the server 20 updates the internal state of the target object (emotion parameters, etc.) based on the outputted cognition and emotion, and stores it in the memory unit. Specifically, the server 20 performs a Sentiment Analysis on the text output from the LLM (e.g., "Very anxious") and quantifies the intensity of a specific emotion (e.g., "Irritability"). Then, based on the calculated intensity, it updates the corresponding emotion parameter stored in the living organism object database 212. In step S1027, the server 20 formulates an action plan to be executed next based on the outputted cognition and the background information and goals of the target object, and generates control data (e.g., destination coordinates, utterance content) to operate the object. At this time, the natural language action plan output from the LLM (e.g., "Go to the cafe") is parsed and converted into a predefined action command (e.g., moveTo(target_location)) and its arguments (e.g., the coordinates of the cafe). For example, if the action plan is "Move to the right-hand aisle to avoid the crowd," the server 20 (control data generation module 2038) calculates the specific movement path and target coordinates using virtual space map information and a pathfinding algorithm (e.g., A* algorithm). Alternatively, the cognitive model (LLM) itself may execute a pathfinding function, etc., by calling an available external function (tool) (e.g., using Function Calling technology) to generate control data.

[0180] In step S1028, the server 20 executes the actions of the target object in the virtual space based on the generated control data.

[0181] Figure 11 shows the process flow for identifying input data for an object.

[0182] In step S1121, the server 20 determines the target object that identifies the input data.

[0183] In step S1122, the server 20 sets up a virtual camera based on the current position and orientation of the target object and generates visual information (image data or semantic information) by imaging (rendering) the virtual space. Semantic information here refers to a description of the type, position, and state of objects within the imaging range of the virtual camera, for example, as text or structured data. Specific examples include data in the format of "[Object: Person, Position: (10, 25), State: Walking], [Object: Traffic Light, Position: (12, 30), State: Red]". This allows the LLM to directly understand the visual situation without analyzing the image itself. If image data is generated, a multimodal model (e.g., VLM (Vision Language Model)) that can handle visual information as a cognitive model can be used to directly use the image information as a basis for judgment. Alternatively, the image data can be converted into linguistic information (semantic information) using a separately prepared image analysis model (e.g., image caption generation model) and then input into the cognitive model.

[0184] In step S1123, the server 20 identifies the environmental conditions (temperature, humidity, brightness, noise level, etc.) of the location where the target object is located from the virtual space data.

[0185] In step S1124, the server 20 generates simulated biometric information (EEG, heart rate, body temperature, sweating, etc.) based on the identified environmental conditions and the current state of the target object (activity level, etc.). This generation is performed, for example, using a predefined rule-based model or mathematical model. For this generation, for example, a predetermined mathematical model (e.g., a thermoregulation model based on physiology) or rule-based logic (e.g., "increase sweating when the temperature is high and the activity level is high") can be used, which takes environmental conditions (e.g., temperature) and activity level (e.g., movement speed) as inputs and outputs the amount of change in biometric information.

[0186] Specifically, the system dynamically calculates simulated biological information based on rules such as "If the 'anxiety' parameter in the object's internal state exceeds 0.7 and the 'movement speed' is 1.5 m / s or more, set the simulated heart rate to 120 beats / minute," and formulas such as "Simulated body temperature = Basal body temperature + (Activity level × Coefficient A) + (Influence from external temperature × Coefficient B)."

[0187] Furthermore, it is possible to use the internal state of an object (e.g., the "anxiety" level of the emotion parameter) as input to reflect psychological effects (such as an increase in heart rate due to stress). The generated simulated biometric information is used as input data for the cognitive model (S1024 in Figure 10), and for example, information such as "heart rate is elevated" influences the interpretation of the situation as "tense" or "anxious."

[0188] In step S1125, the server 20 identifies information within the receivable range of the actions (movement status, speech content, etc.) of other objects present around the target object and acquires it as other object information.

[0189] In step S1126, the server 20 determines, based on the running scenario data, whether there are any events affecting the target object (e.g., disaster notification, path blockage information), and if so, identifies that information.

[0190] In step S1127, the server 20 integrates all the information generated or identified in steps S1122 to S1126 (visual, environmental, simulated organism, other objects, and scenario information) to create input data for the cognitive model.

[0191] Figure 12 shows the flow of processes in urban development and spatial design.

[0192] In step S1211, terminal 10 accepts the user's (designer's) selection of virtual space data (such as base terrain) to be evaluated.

[0193] In step S1212, terminal 10 accepts the user's definition of a proposed layout plan for buildings, plazas, roads, etc., in the virtual space (placement of 3D models, changes to settings, etc.).

[0194] In step S1213, terminal 10 accepts natural language input from the user regarding their requests for the space (e.g., "I want it to feel like a hotel") or the setting of a target psychological state (e.g., "Maximize comfort") as an AI-assisted function.

[0195] In step S1221, the server 20, as an AI-assisted function, generates recommended values ​​(e.g., proposed layout, proposed design) for the optimal spatial arrangement or design based on the requests and target states received from the terminal 10. For this generation, for example, generative AI technology (e.g., design generation using diffusion models), or optimization algorithms such as reinforcement learning or genetic algorithms can be used to search for and generate arrangement parameters and design elements that maximize the input requests and target states.

[0196] In step S1214, terminal 10, as an AI-assisted function, presents the recommended values ​​generated by server 20 to the user and accepts user input to adopt the recommended values ​​as a proposed layout plan.

[0197] In step S1215, terminal 10 accepts the user's settings for simulation conditions (time of day, season, weather conditions, attributes and number of objects to be placed, etc.) and sends an execution instruction to server 20.

[0198] In step S1222, the server 20 performs a simulation (see Figure 9) based on the received deployment plan and simulation conditions.

[0199] In step S1223, the server 20 collects and records data on the visual experience of objects, movement patterns (people flow), and psychological states (pleasant / unpleasant, sense of crowding) during the simulation, and calculates evaluation indices. Specifically, it collects emotional parameters (see Figure 7; e.g., "joy," "anxiety") and cognitive content (e.g., "It's crowded and unpleasant") recorded as the internal state of each object during the simulation. Then, it aggregates this data for specific areas (e.g., meshes) and time periods in which the virtual space is divided, and performs statistical processing (e.g., mean, variance, percentage of objects exceeding a specific threshold) to calculate quantitative evaluation indices such as the "average comfort score in the area" and the "stress level distribution by time period."

[0200] In step S1224, the server 20 analyzes the simulation results and generates an evaluation report of the proposed layout plan (e.g., bottlenecks in pedestrian flow, analysis of landscape changes by time of day and season). This analysis includes evaluating the impact of visual and environmental changes associated with changes in time of day, season, and weather (e.g., changes in sunlight conditions, temperature, vegetation, and nighttime lighting effects) on the behavior of objects. For example, in the daytime during summer, in a plaza with little shade and high temperatures, objects may be more likely to avoid the area (perception of discomfort based on visual and environmental information). Also, during rainy weather or at night, objects may feel uneasy (perception based on visual information) and avoid using dark paths with insufficient lighting. This system identifies these changes as input data (Figure 11) and simulates changes in object behavior through a cognitive model, enabling a detailed analysis of changes in spatial use according to time, season, and weather.

[0201] Furthermore, Server 20 can compare the results obtained from the simulation (e.g., the number of people passing through a specific area at different times of day) with real-world data observed in the real world (e.g., actual passerby data from cameras installed in the same area) and calculate the error. Based on this comparison, the user can adjust (calibrate) the background information of the living organism object and the parameters of the cognitive model (e.g., outing tendencies, degree of crowd avoidance) and run the simulation again, thereby gradually improving the prediction accuracy of the model. This increases the reliability of the system as an analytical tool that more faithfully reproduces the real world.

[0202] In step S1216, terminal 10 receives an evaluation report from server 20 and presents it to the user by combining a heat map, movement diagram, viewpoint video (experience of changes according to time and season).

[0203] <4 Screen Examples> Figure 13 is an example of an operation screen that provides an overview of how numerous objects move and act autonomously in a virtual space.

[0204] The simulation operation screen 1300 is an operation screen that provides an overview of the entire virtual space and allows users to observe and manipulate the autonomous activities of numerous objects.

[0205] The virtual space 3D view display area 1302 is the main area that provides an overview of the entire virtual space (terrain, buildings) and the numerous objects (agents) operating within it.

[0206] The simulation status display area 1304 is an area that displays the current simulation time, execution status (running / stopped), execution speed, total number of agents, etc.

[0207] The object details display area 1306 is a panel that displays detailed information (background information, current status, etc.) about a specific selected object.

[0208] The interaction visualization display area 1308 is an area that visualizes and displays communication between objects on the 3D view using callouts and icons.

[0209] The viewpoint control UI1310 is an operation unit consisting of mouse operations and a group of buttons for controlling the viewpoint of the 3D view (camera position, angle, zoom level).

[0210] The object selection operation unit 1312 is an operation unit for selecting an object to display detailed information about by performing operations such as clicking on an object in the 3D view.

[0211] The simulation control unit 1314 is a group of buttons that control the playback, pause, and stop of the simulation, and is an operation unit that accepts user input. While the simulation is paused, the user can select a specific object or area and dynamically provide new information (for example, emergency news via the initial information input UI 1514 in Figure 15). By then resuming the simulation, the impact of the intervention on the surroundings can be observed immediately.

[0212] The execution speed adjustment unit 1316 is a slider or input field for adjusting the simulation's progress speed (e.g., normal speed, 10x speed), and is an operation unit that accepts specifications.

[0213] The display layer switching unit 1318 is a switch that toggles the display of layers of information to be displayed (e.g., terrain, buildings, agents, analysis results), and is an operation unit that accepts specifications.

[0214] Figure 14 shows an example of an operation screen that displays the viewpoint of a specific object and detailed information about that object.

[0215] The object viewpoint / state manipulation screen 1400 is an operation screen that displays the viewpoint (virtual camera image) of a specific object and its internal information in detail.

[0216] The virtual space 3D view display area 1402 is the area that displays the image of the virtual space as seen from the viewpoint (virtual camera) of the selected object.

[0217] The background information display area 1404 is an area that displays fixed attribute information of an object, such as its personality, background, social attributes, and income.

[0218] The internal state display area 1406 is an area that displays the dynamic internal state of an object, such as its current emotions, cognitive content, and physiological needs.

[0219] The simulated biometric information display area 1408 is an area that displays the object's current simulated biometric information (heart rate, body temperature, sweating, etc.).

[0220] The behavior log display area 1410 is an area that displays the past actions and cognitive changes performed by the target object in chronological order.

[0221] The target object selection section 1412 is a list or pull-down menu for selecting and switching the target object for displaying the viewpoint or state, and is an operation section that accepts specifications.

[0222] The viewpoint switching section 1414 consists of buttons and tabs for switching the display mode of the 3D view (first-person view, follow view, overhead view, etc.), and is an operation section that accepts user input.

[0223] Figure 15 is an example of an operation screen that shows how multiple objects communicate with each other (through chat and voice communication), how information spreads, and how their behavior changes.

[0224] The multi-agent interaction operation screen 1500 is an operation screen that visualizes how multiple objects communicate with each other, how information is propagated, and how their behavior changes.

[0225] The virtual space view display area 1502 is the area that displays the virtual space (physical space or information space (SNS, etc.)) in which an object operates. When simulating an information space, the virtual space is defined not by a physical geographic coordinate system, but by, for example, a network structure (e.g., SNS follow relationships, community affiliation relationships). Objects have an address in the information space (e.g., account ID), and instead of physical movement, they perform communication actions such as posting, viewing, and sharing messages. In this case, the input data consists of messages from others and information on the timeline, and the cognitive model decides on a response (reply, spread, change of opinion, etc.) based on this data.

[0226] The communication visualization display area 1504 is an area that displays the content of information (chat, voice) transmitted by an object in a speech bubble format or similar near the object on the view.

[0227] The communication log display area 1506 is an area that displays the history of communications that have occurred (who communicated what to whom) in chronological order.

[0228] The information propagation network diagram display area 1508 is an area that visualizes the information propagation paths and scope of influence as a network diagram.

[0229] The behavior change highlighting area 1510 is an area that highlights (changes color, displays icons, etc.) objects whose behavior has changed as a result of receiving information.

[0230] The display filter setting unit 1512 is a setting UI for filtering the types of interactions to display (e.g., specific topics, interactions between specific agents), and is an operation unit that accepts specifications.

[0231] The initial information input UI1514 is an input interface for intentionally generating specific information (e.g., misinformation, news) during the simulation.

[0232] Figure 16 is an example of an operation screen showing how objects take evacuation actions based on their perception in a disaster scenario where routes are blocked.

[0233] Disaster simulation operation screen 1600 is an operation screen that shows a disaster scenario occurring and objects taking evacuation actions.

[0234] The virtual space 3D view display area 1602 is an area that visualizes and displays the occurrence and affected area of ​​disasters (fire, flood, etc.).

[0235] The route closure status display area 1604 is an area that highlights routes and areas that have become impassable due to a disaster (e.g., by displaying warning colors or icons).

[0236] The evacuation behavior visualization display area 1606 is an area that visualizes how objects are performing evacuation actions based on their perception (such as movement routes, speed, and status icons).

[0237] The simulation result indicator display area 1608 is an area that displays the progress of the scenario (elapsed time) and statistical information on the evacuation situation (evacuation completion rate, number of people stranded, etc.).

[0238] The scenario selection unit 1610 is a menu for selecting a disaster scenario to execute, and is an operation unit that accepts the selection.

[0239] The disaster occurrence condition setting screen 1612 is used to set conditions such as the location, scale, and time of a disaster.

[0240] The simulation start operation unit 1614 is a button for starting a simulation in the selected and set scenario, and is an operation unit that accepts input.

[0241] Figure 17 shows an example of an operation screen that displays simulation results visualizing pedestrian flow, congestion, and psychological state in a specific layout plan.

[0242] The Urban Planning / Pedestrian Flow Simulation Operation Screen 1700 is an operation screen that displays simulation results that visualize pedestrian flow, congestion, and psychological state in a specific layout plan.

[0243] The virtual space 3D view display area 1702 is the area that displays the virtual space to which the evaluation target layout plan (buildings, plazas, roads, etc.) has been applied.

[0244] The pedestrian flow / movement diagram display area 1704 is an area that displays the movement trajectories of objects and the overall flow based on the simulation results, overlaid as a movement diagram.

[0245] The dwell time heatmap display area 1706 is an area that displays the locations and densities where objects tend to linger as a heatmap.

[0246] The psychological state distribution display area 1708 is an area that visualizes the distribution of the psychological state of objects (such as crowding, comfort level, pleasant / unpleasant) using color coding or other methods. In the illustrated example, the distribution of the psychological state of objects in an area is displayed in the same area as the dwell time heatmap display area 1706, for example, in response to user operation (or automatically switched without user operation). However, both the dwell time heatmap display area 1706 and the psychological state distribution display area 1708 may be displayed on the screen together.

[0247] The evaluation indicator display area 1710 is a dashboard area that displays quantitative evaluation indicators such as the number of users in a specific area, the average time spent there, and the results of analysis of bottlenecks. In the evaluation indicator display area 1710, the distribution of the psychological state of objects (such as crowding, comfort level, pleasant / unpleasant) may also be displayed, as explained in the psychological state distribution diagram display area 1708.

[0248] The layout plan selection unit 1712 is a UI for selecting and switching between layout plans to be compared and evaluated (e.g., Plan A, Plan B), and is an operation unit that accepts specifications.

[0249] The evaluation indicator selection unit 1714 is a tab or menu for selecting and switching the evaluation indicators and visualization modes (such as pedestrian flow, dwell time, and psychological state) to be displayed, and is an operation unit that accepts specifications.

[0250] Figure 18 shows an example of an operation screen that displays a comparison of visual changes in the same virtual space under different time periods, seasons, and weather conditions.

[0251] The Time / Season Change Operation Screen 1800 is an operation screen that displays a comparison of visual changes in the same virtual space under different time periods, seasons, and weather conditions.

[0252] The virtual space 3D view display area 1802 is an area that displays the scenery of the virtual space under set conditions (time, season, weather), and it is possible to display multiple conditions side by side for comparison or to switch between them.

[0253] The sunlight and shadow simulation display area 1804 is an area that visualizes changes in sunlight conditions and shadows according to the position of the sun based on the set conditions.

[0254] The currently set conditions display area 1806 is the area that displays the environmental conditions (date and time, season, weather) of the currently displayed view.

[0255] The viewpoint setting UI1808 is a user interface for setting, saving, and recalling the position and orientation of the viewpoint used to view the scenery.

[0256] The time zone setting unit 1810 is a slider UI for changing the time zone of the simulation, and is an operation unit for reflecting the changes in the 3D view in real time.

[0257] The season / date selection section 1812 is a calendar or menu for selecting the season or a specific date for the simulation, and is an operation section that accepts specifications.

[0258] The weather condition selection unit 1814 is a menu for selecting weather conditions (sunny, rainy, snowy, etc.) and is an operating unit that accepts the specified conditions.

[0259] The tour playback control unit 1816 is a button for playing back a series of pre-set conditional changes (e.g., changes over a day, changes over the seasons) as an animation, and is an operation unit that accepts input.

[0260] Figure 19 shows an example of a user interface (UI) where the user inputs requests in natural language, and an operation screen displaying recommended design proposals generated by AI based on those inputs.

[0261] The AI ​​spatial design support operation screen 1900 is an operation screen where the user inputs requests in natural language, and the AI ​​generates and displays recommended design proposals based on those requests.

[0262] The virtual space 3D view display area 1902 is the area that displays a 3D preview of the currently selected or AI-generated design proposal (spatial layout, interior, etc.).

[0263] The AI ​​Recommended Design List Display Area 1904 is an area that displays multiple recommended design proposals (recommended values) generated by the AI ​​in a list format (with thumbnails and evaluation scores).

[0264] The recommendation reason / evaluation score display area 1906 is the area that displays the reasons why the AI ​​recommends the selected proposal, as well as an evaluation score relative to the target state.

[0265] The request input section 1908 is a text box for users to input their requests regarding the space (e.g., "I want it to feel like a hotel," "I want it to be open") in natural language.

[0266] The target state setting UI1910 is a user interface for setting the psychological and physical state of the target object (e.g., "Maximize comfort," "Prioritize work efficiency").

[0267] The recommendation generation execution unit 1912 is a button that initiates the generation of recommended values ​​(design proposals) by AI based on the input requests and target states, and is an operation unit that accepts user input.

[0268] The recommended application unit 1914 is a button for applying the recommended option selected from the list to the 3D view (virtual space), and is an operation unit that accepts user input.

[0269] The feedback input UI1916 is a user interface for providing evaluations of the generated recommendations, inputting additional correction instructions, and requesting regeneration.

[0270] <Variation>

[0271] (1) Information processing system that performs language processing in response to prompt input In the above description of the embodiment, as an example of an information processing system that performs language processing in response to prompt input, an example was described in which a prompt is sent to a server 95 of a large-scale language model service, which is an external information processing system, and the result generated by the server 95 of the large-scale language model service in response to the prompt is received.

[0272] Alternatively, instead of using an external information processing system, a large-scale language model may be run on terminal 10 to perform processing in response to prompts. Alternatively, the large-scale language model may be run on server 20, which holds the information to be included in the prompts.

[0273] As described above, the information processing system that performs language processing in response to prompt input may be an external information processing system different from the service provided by the server 20, or the server 20 may have the functions of the information processing system, or it may be operated on the terminal 10. The terminal 10 may be provided with a processor such as a GPU (Graphics Processing Unit) for operating a large-scale language model, and the processor may cause the large-scale language model to perform the function of performing language processing in response to prompt input. For example, the server 20 may provide a large-scale language model to be operated on the terminal 10, and the terminal 10 may construct an information processing system that performs language processing.

[0274] <Others> In the above description, the "processor" refers to one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

[0275] Further, the at least one processor may be a processor in a broad sense such as a hardware circuit (for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0276] Further, in the above description, expressions such as "xxx table" may be used to describe information that allows an output to be obtained for an input, but this information may be data of any structure, or may be a learning model such as a neural network that generates an output for an input. Therefore, the "xxx table" can be referred to as "xxx information".

[0277] Furthermore, the above description is merely an example of the structure of each table; one table may be divided into two or more tables, or all or part of two or more tables may constitute a single table.

[0278] Furthermore, while the above explanation sometimes uses "program" as the subject to describe the process, a program is executed by a processor and performs defined processes using the memory and / or interface as appropriate. Therefore, the subject of the process may also be the processor (or a device such as a controller that has that processor).

[0279] The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0280] A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0281] Furthermore, by distributing all or part of each hardware configuration across multiple computers and connecting them to each other via a network, a computer can be virtually realized. Thus, the concept of a computer includes not only computers housed in a single enclosure or case, but also virtualized computer systems.

[0282] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0283] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0284] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0285] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0286] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0287] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0288] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0289] In the above embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, the collection of each device can be considered as a single "information processing device," and the system may be formed as a collection of multiple devices. The way in which the multiple functions required to realize the system according to this embodiment are allocated to one or more hardware can be appropriately determined in consideration of the processing capacity of each hardware and / or the specifications required for the system.

[0290] In the above embodiment, the server may distribute a program to be executed on the terminal in the service provided by the server, and the terminal may perform the processing after installing the program, or the processing may be appropriately divided between the terminal and the server. For example, each of the processes in the above embodiment can be performed on the terminal or on the server.

[0291] The embodiments described above do not unduly limit the scope of the disclosure as defined in the claims. Furthermore, not all components shown in the embodiments are essential components of the disclosure. Also, the figures are schematic and not necessarily strictly illustrative.

[0292] Furthermore, while identification numbers are used as identification information for various objects in the above explanation, other types of identification information (for example, identifiers including letters or symbols) may also be used.

[0293] Furthermore, in the above explanation, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0294] Furthermore, in the above explanation, the control lines and information lines shown are those deemed necessary for the explanation, and do not necessarily represent all control lines and information lines in the actual product. All components may be interconnected.

[0295] (Supplementary Notes) The matters described in the foregoing respective embodiments are supplemented below.

[0296] (Supplementary Note 1) A program for operating a computer comprising one or a plurality of computer processors, the program being configured to store, in a storage unit, data of a virtual space that reproduces a real space and information of an object representing a living organism, wherein a model that outputs a response in accordance with data related to nerves of the living organism is set for the object representing the living organism, and the program causes the one or the plurality of computer processors to execute a step of defining the virtual space that reproduces the real space, a step of arranging the object representing the living organism in the virtual space, a step of identifying input data that exerts an influence on the object representing the living organism based on data set in the virtual space, a step of generating control data for the object representing the living organism based on output data output from the model by inputting the input data identified in the identifying step to the model set for the object, and a step of causing the object representing the living organism to operate in the virtual space based on the control data generated in the generating step.

[0297] (Supplementary Note 2) The program according to Supplementary Note 1, wherein the data of the virtual space includes terrain data, an environment model, and physical law data, and in the identifying step, information on a result of a natural phenomenon reproduced in the virtual space in accordance with a physical law is identified as the input data.

[0298] (Supplementary Note 3) The program according to any one of Supplementary Notes 1 to 2, wherein in the generating step, the output data output from the model includes cognitive data indicating cognition of the living organism, and the control data is generated based on the cognitive data as the generation of the control data.

[0299] (Supplementary Note 4) In the generation step, the cognitive data includes data indicating the emotions of the living organism, and the program generates control data based on the data indicating the emotions of the living organism as cognitive data, as described in Appendix 3.

[0300] (Note 5) In the memory unit, objects representing living organisms are stored with background information including personality, history, and social attributes, and in the generation step, control data is generated, and the control data is generated based on the background information, as described in any of the programs in Appendix 1 to 4.

[0301] (Note 6) Background information, including income-related information, is provided in the program described in Appendix 5.

[0302] (Note 7) A program described in any of the appendices 1 to 6, which, in the generation step, takes input data as a model and inputs it into a large-scale language model to obtain output data from the large-scale language model.

[0303] (Note 8) In the identification step, the program described in any of Appendix 1 to 7 includes, as input data, information corresponding to the vision of a living organism, generated from imaging information of a virtual camera that images a virtual space based on the position and orientation of an object.

[0304] (Note 9) A program as described in any of Appendix 1 to 8, wherein, in the identification step, the input data includes simulated data corresponding to at least one biometric information selected from the group consisting of electroencephalogram, heart rate, body temperature, and sweating, which is generated based on environmental conditions including weather at the location where the object is located.

[0305] (Note 10) The program described in one of the appendices 1 to 9 contains objects representing multiple living organisms in a virtual space, and these objects operate autonomously based on the models set for each of them.

[0306] (Note 11) The program described in Appendix 10, wherein each of multiple objects can send information to other objects via chat or voice, and in the identification step, the input data to a certain object includes information sent from other objects as input data.

[0307] (Note 12) The program is one of the programs described in any of Appendix 1 to 11, further comprising the step of causing one or more computer processors to perform the step of setting up a scenario that simulates an event including at least one of a disaster or an accident in a virtual space, and in the step of identifying input data, the program includes identifying input data based on the event set up in the scenario.

[0308] (Note 13) The scenario includes a setting that blocks the paths an object can travel due to an event, and the object moves based on the path blocking setting and the cognitive data output from the model, as described in Appendix 12.

[0309] (Note 14) A virtual space is a program described in any of the appendices 11 to 13, which replicates an information network including social networking services.

[0310] (Note 15) The program is one of the programs described in any of Appendix 1 to 14, which further outputs control signals to one or more computer processors for controlling real-world equipment or systems based on the results of a simulation in which an object is operated.

[0311] (Note 16) The program is one of the programs described in any of Appendix 1 to 15, which causes one or more computer processors to further perform the steps of defining a layout plan for at least one of buildings, plazas, and roads in a virtual space, and simulating at least one of the visual experience, movement patterns, and psychological states of objects representing living beings in the virtual space based on the defined layout plan.

[0312] (Note 17) The program described in Appendix 16, which in the simulation step reflects the visual changes of the virtual space in response to changes in at least one of the time of day, season, and weather conditions, and simulates the effect of such visual changes on the visual experience of objects.

[0313] (Note 18) The program is one of the programs described in any of Appendix 1 to 17, which causes one or more computer processors to perform at least one of the following steps: receiving input from a user in natural language regarding a space to be represented in a virtual space, or receiving settings for the psychological or physical state of an object representing a living being; and generating recommended values ​​for the arrangement or design of the space in the virtual space based on at least one of the requests in natural language or the settings for the psychological or physical state.

[0314] (Note 19) A method performed by a computer having one or more computer processors, wherein a memory unit is configured to store data of a virtual space that reproduces a real space and information of an object representing a living organism, and the object representing a living organism is set up with a model that outputs a response in accordance with data related to the nerves of the living organism, and the method is a method in which one or more computer processors perform the steps of: defining a virtual space that reproduces a real space; placing an object representing a living organism in the virtual space; identifying input data that affects the object representing a living organism based on data set up in the virtual space; generating control data for the object representing a living organism based on output data output from the model by inputting the input data identified in the identification step into the model set up on the object; and operating the object representing a living organism in the virtual space based on the control data generated in the generation step.

[0315] (Note 20) An information processing device comprising a control unit, wherein the memory unit is configured to store data of a virtual space that reproduces a real space and information of an object representing a living organism, the object representing a living organism is set up with a model that outputs a response in accordance with data related to the nerves of the living organism, and the control unit performs the steps of: defining a virtual space that reproduces a real space; arranging an object representing a living organism in the virtual space; identifying input data that affects the object representing a living organism based on the data set up in the virtual space; inputting the input data identified in the identification step into the model set up on the object and generating control data for the object representing a living organism based on the output data output from the model; and operating the object representing a living organism in the virtual space based on the control data generated in the generation step.

Claims

1. A program for operating a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The program is configured on one or more computer processors. The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of making an object representing the living organism operate in the virtual space based on the control data generated in the above generation step, A program in which, in the identifying step, the input data includes information corresponding to the vision of the living organism, generated from imaging information of a virtual camera that images the virtual space based on the position and orientation of the object.

2. A program for operating a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The program is configured on one or more computer processors. The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of making an object representing the living organism operate in the virtual space based on the control data generated in the above generation step, A program that, in the identifying step, includes as input data simulated data corresponding to at least one biometric information selected from the group consisting of electroencephalogram, heart rate, body temperature, and sweating, which is generated based on environmental conditions including weather at the location where the object is located.

3. A program for operating a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The program is configured on one or more computer processors. The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of making an object representing the living organism operate in the virtual space based on the control data generated in the above generation step, The program is configured on one or more computer processors. The steps include defining a layout plan for at least one of buildings, plazas, and roads in the virtual space, A step of simulating at least one of the visual experience, movement patterns, and psychological state of an object representing the living organism in the virtual space, based on the defined placement plan; Let's execute it further, A program that, in the simulation step, reflects the visual changes of the virtual space in response to changes in at least one of the time of day, season, and weather conditions, and simulates the effect of such visual changes on the visual experience of the object.

4. A program for operating a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The program is configured on one or more computer processors. The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of making an object representing the living organism operate in the virtual space based on the control data generated in the above generation step, In the generation step, the output data output from the model includes cognitive data indicating the cognition of the living organism. To generate the aforementioned control data, the control data is generated based on the aforementioned cognitive data, The program further provides the following to one or more computer processors: The step of setting up a scenario in the virtual space that simulates an event including at least one of a disaster or an accident is further performed. In the step of identifying the above, identifying the input data includes identifying the input data based on the events set in the scenario, The aforementioned scenario includes a setting that blocks the passable path of the object due to the aforementioned event, The object is a program that moves based on the setting of blocking the path and the cognitive data output from the model.

5. The data of the aforementioned virtual space includes terrain data, environmental models, and physical law data. The program according to any one of claims 1 to 4, wherein in the step of identifying, information regarding the results of natural phenomena reproduced in the virtual space according to the physical law data is identified as the input data.

6. In the generation step, the output data output from the model includes cognitive data indicating the cognition of the living organism. The program according to any one of claims 1 to 3, wherein the program generates the control data based on the cognitive data.

7. In the generation step, the cognitive data includes data indicating the emotions of the living organism. The program according to claim 6, wherein the control data is generated based on data indicating the emotions of the living organism as cognitive data.

8. In the memory unit, the object representing the living organism is stored with background information including personality, history, and social attributes. The program according to any one of claims 1 to 4, wherein, in the step of generating, the program generates the control data based on the background information.

9. The program according to claim 8, wherein the background information includes information relating to revenue.

10. The program according to any one of claims 1 to 4, wherein in the generation step, the program obtains the output data output from the large-scale language model by inputting the input data as the model into the large-scale language model.

11. The program according to any one of claims 1 to 4, wherein a plurality of objects representing the living organisms are placed in the virtual space, and the plurality of objects operate autonomously based on the model set for each of them.

12. Each of the aforementioned objects can send information to other objects via chat or voice. The program according to claim 11, wherein, in the step of identifying, the input data for a certain object includes information transmitted from another object in the input data.

13. The program according to claim 12, wherein the virtual space is a reproduction of an information network including a social networking service.

14. The program further provides the following to one or more computer processors: The program according to any one of claims 1 to 4, which outputs a control signal for controlling a real-world device or system based on the results of a simulation in which the aforementioned object is operated.

15. The program is configured on one or more computer processors. The process includes at least one of the following steps: receiving input from the user in natural language regarding requests for the space to be represented in the virtual space, or receiving settings for the psychological or physical state of an object representing a living being; A step of generating recommended values ​​regarding the arrangement or design of space in the virtual space based on the request in natural language or the setting of the psychological or physical state, A program according to any one of claims 1 to 4, which further executes the following.

16. A method performed by a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The above method involves one or more computer processors, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, A method comprising the identifying step, wherein the input data includes information corresponding to the vision of the living organism, generated from imaging information of a virtual camera that images the virtual space based on the position and orientation of the object.

17. A method to be executed by a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The above method involves one or more computer processors, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, A method comprising the identifying step, wherein the input data includes simulated data corresponding to at least one biometric information selected from the group consisting of electroencephalogram, heart rate, body temperature, and sweating, generated based on environmental conditions including weather at the location where the object is located.

18. A method to be executed by a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The above method involves one or more computer processors, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, The above method involves one or more computer processors, The steps include defining a layout plan for at least one of buildings, plazas, and roads in the virtual space, A step of simulating at least one of the visual experience, movement patterns, and psychological state of an object representing the living organism in the virtual space, based on the defined placement plan; Let's execute it further, A method for simulating the visual changes of the virtual space in response to changes in at least one of the time of day, season, and weather conditions, and for simulating the effect of such visual changes on the visual experience of the object.

19. A method to be executed by a computer having one or more computer processors, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The above method involves one or more computer processors, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, In the generation step, the output data output from the model includes cognitive data indicating the cognition of the living organism. To generate the aforementioned control data, the control data is generated based on the aforementioned cognitive data, The above method further involves the one or more computer processors, The step of setting up a scenario in the virtual space that simulates an event including at least one of a disaster or an accident is further performed. In the step of identifying the above, identifying the input data includes identifying the input data based on the events set in the scenario, The aforementioned scenario includes a setting that blocks the passable path of the object due to the aforementioned event, A method by which the object moves based on the setting of blocking the path and the cognitive data output from the model.

20. An information processing device comprising a control unit, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The control unit, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, An information processing device in which, in the step of identifying, the input data includes information corresponding to the vision of the living organism, generated from imaging information of a virtual camera that images the virtual space based on the position and orientation of the object.

21. An information processing apparatus comprising a control unit, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The control unit, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, An information processing device in which, in the identifying step, the input data includes simulated data corresponding to at least one piece of biometric information selected from the group consisting of electroencephalogram, heart rate, body temperature, and sweating, which is generated based on environmental conditions including the weather at the location where the object is located.

22. An information processing device comprising a control unit, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The control unit, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, The control unit, The steps include defining a layout plan for at least one of buildings, plazas, and roads in the virtual space, A step of simulating at least one of the visual experience, movement patterns, and psychological state of an object representing the living organism in the virtual space, based on the defined placement plan; Further execution, An information processing device that, in the simulation step, reflects the visual changes of the virtual space in response to changes in at least one of the time of day, season, and weather conditions, and simulates the effect of such visual changes on the visual experience of the object.

23. An information processing apparatus comprising a control unit, The memory unit is configured to store data of a virtual space that reproduces the real space, and information of objects representing living organisms. The objects representing living organisms are set up with a model that outputs a response according to the data related to the living organism's nervous system. The control unit, The steps include defining the virtual space that reproduces the real space, The steps include: placing an object representing the living organism in the virtual space; The steps include identifying input data that affects an object representing a living organism based on data set in the virtual space, A step of inputting the input data identified in the identification step into the model set for the object, thereby generating control data for the object representing the living organism based on the output data output from the model, The steps include: Executing the step of operating an object representing the living organism in the virtual space based on the control data generated in the above generation step, In the generation step, the output data output from the model includes cognitive data indicating the cognition of the living organism. To generate the aforementioned control data, the control data is generated based on the aforementioned cognitive data, The control unit, The step of setting up a scenario in the virtual space that simulates an event including at least one of a disaster or an accident is further performed. In the step of identifying the above, identifying the input data includes identifying the input data based on the events set in the scenario, The aforementioned scenario includes a setting that blocks the passable path of the object due to the aforementioned event, The object is an information processing device that moves based on the setting of blocking the path and the cognitive data output from the model.

Images