Information processing device, information processing method, and information processing program

Abstract

This invention provides an information processing device, an information processing method, and an information processing program that can provide a diverse virtual space participation-based mobility environment that takes into account the prediction of human behavior in relationships with others, i.e., interactions. [Solution] The information processing device comprises a measurement unit that measures the movement simulations of multiple users in real space and reflects the measured movement simulation results in a virtual space; a storage unit that stores the movement status of multiple users; a model generation unit that generates simulation model data based on the movement status and stored data; and a simulation execution unit that executes a simulation based on the model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation.

Description

Technical Field

[0005] , , , ,

[0004] , ,

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program, and particularly to an information processing apparatus, an information processing method, and an information processing program that provide a diverse virtual space participation type of mobile environment.

Background Art

[0002] A technology called Digital Twin has been conventionally known. Digital Twin is used mainly in the manufacturing field and the like, and is digital information that reproduces a real object (for example, a machine part or the like) in a digital space. By using Digital Twin, it becomes possible to grasp in real time events occurring in a real object, etc., and thus it is utilized, for example, in improving the quality of products and the like.

[0003] It is considered that by expanding the target range of Digital Twin to objects and organisms that do not actually exist, and performing simulations and the like using these Digital Twins, it is possible to support new value creation and problem solving of various problems. For example, the information processing apparatus, information processing method, and information processing program disclosed in Patent Document 1 are said to be capable of estimating relevant parameters regarding human behavior.

[0004] In the technology disclosed in Patent Document 1, information of a real organism (including a person) is acquired, and the acquired information is output into a sandbox, and communication with a virtual Digital Twin (DT) can be achieved in the sandbox (communication between a person and a virtual DT in a virtual space). However, the relationship with others, that is, human behavior prediction in interaction (interactive action) is not considered, and it cannot be said that it sufficiently meets social needs.

Prior Art Documents

[0006] Therefore, the present invention aims to provide an information processing device, an information processing method, and an information processing program that can provide a diverse virtual space participation mobility environment involving multiple entities, taking into account the prediction of human behavior in relationships with others, i.e., interactions. [Means for solving the problem]

[0007] In other words, the information processing device according to the first embodiment includes: a measurement unit that measures the movement simulations of multiple users in real space and reflects the measured movement simulation results in a virtual space; a storage unit that stores the movement status of multiple users; a model generation unit that generates simulation model data based on the movement status and stored data; and a simulation execution unit that executes a simulation based on the model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation.

[0008] A second embodiment may be an information processing device according to the first embodiment in which multiple users can participate in the virtual space using multiple types of means of transportation, including vehicles, pedestrians, and cyclists.

[0009] A third embodiment is an information processing device according to the first embodiment, in which the model generation unit reproduces traffic conditions in a virtual space based on parameters including the proportion of pedestrians and vehicles mixed together and the proportion of users with different personalities and physical characteristics mixed together.

[0010] A fourth embodiment may further include, in the information processing device according to the first embodiment, an acquisition unit for collecting simulation results, an storage unit for storing the results collected by the acquisition unit, and an analysis unit for analyzing the results stored by the storage unit.

[0011] The information processing method relating to the fifth aspect causes a computer to perform a measurement step of measuring the movement simulations of multiple users and reflecting the measured movement simulation results in a virtual space; a storage step of storing the movement status of multiple users; a model generation step of generating simulation model data based on the movement status and stored data; and a simulation execution step of executing a simulation based on the model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation.

[0012] The information processing program according to the sixth aspect includes a measurement function that measures the movement simulations of multiple users and reflects the measured movement simulation results in a virtual space; a storage function that stores the movement status of multiple users; a model generation function that generates simulation model data based on the movement status and stored data; and a simulation execution function that executes a simulation based on the model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation. [Effects of the Invention]

[0013] The information processing device according to the present invention is characterized by comprising: a measurement unit that measures the movement simulations of multiple users in real space and reflects the measured movement simulation results in a virtual space; a storage unit that stores the movement status of multiple users; a model generation unit that generates simulation model data based on the movement status and stored data; and a simulation execution unit that executes a simulation based on the model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation. Therefore, it can provide a diverse virtual space participation type movement environment with multiple entities that takes into account the prediction of human behavior in relationships with others, i.e., interactions.

[0014] Furthermore, the information processing method and information processing program according to the present invention, like the information processing apparatus according to the present invention described above, can provide a diverse virtual space participation-type mobility environment involving multiple entities, taking into account the prediction of human behavior in relationships with others, i.e., interactions. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a diagram illustrating the usage of the information processing device according to this embodiment. [Figure 2] Figure 2 shows the usage status of the information processing device according to this embodiment. [Figure 3] Figure 3 shows the usage status of the information processing device according to this embodiment. [Figure 4] Figure 4 is a block diagram showing the system configuration of the information processing device according to this embodiment. [Figure 5] Figure 5 is a diagram illustrating an interactive simulator in a space where pedestrians and vehicles coexist, according to the information processing device of this embodiment. [Figure 6] Figure 6 is a block diagram showing the functions of the information processing device according to this embodiment. [Figure 7] Figure 7 is an example of a flowchart of the information processing program according to this embodiment. [Modes for carrying out the invention]

[0016] An example of an information processing device 10 according to one embodiment of the present disclosure will be described with reference to Figures 1 to 6. The information processing device 10 is a so-called computer, and is also known as a server, personal computer (hereinafter referred to as PC), notebook PC, tablet PC, or smartphone.

[0017] (Overview of the information processing device 10) First, referring to FIG. 1, the usage status of the information processing apparatus 10 will be described. Here, the traffic and human behavior in the physical space 14 are realistically reproduced in the cyber space 13.

[0018] The information processing apparatus 10 (see FIG. 5) can create a situation where vehicles 1, bicycles 2, pedestrians 3, and new mobility 4 coexist in the virtual space 13, and can measure human behavior through the experience in the virtual space 13. The information processing apparatus 10 can provide a measurement environment in which a plurality of people can recognize each other in the virtual space 13 using different means of transportation (avatars 6). That is, the information processing apparatus 10 can construct an environment in which the reactions of people between different means of transportation can be measured by a plurality of people participating in the virtual space 13 using simulators (simulation devices) of different means of transportation and experiencing a reproducible pseudo-experience. The new mobility 4 refers to new means of transportation, such as electric kick scooters, manned drones, etc., which have newly emerged in recent years. The new mobility simulator 4a is a device that simulates the behavior and reaction of this moving body under various road conditions and driving conditions for new means of transportation.

[0019] Specifically, the information processing apparatus 10 can link all of the human judgment timing and the positional relationship of surrounding traffic participants, can measure what the line of sight is looking at, and by associating the questionnaire to the person who participated as the avatar 6 with the behavior data thereof, receptivity and psychological analysis become possible, and it is possible to evaluate (risk and feeling) as an autonomous vehicle and other traffic participants, and further to experience traffic safety from each position and to evaluate variation and receptivity.

[0020] The avatar 6 is an object that concretizes other subjects who are participating in the virtual space in real time using other information processing apparatuses in the virtual space, and moves in the virtual space under the control of other subjects.

[0021] Model 7 is an object that learns the behavioral patterns of other subjects based on the behavioral information of other subjects who have previously participated in the virtual space, and then acts autonomously in the virtual space. Alternatively, Model 7 may be an object that learns its own behavioral patterns based on its own behavioral information from previous participation in the virtual space, and then acts autonomously in the virtual space. This model is a mathematical model.

[0022] Simulation refers to the act of carrying out something in a simulated manner, and in this embodiment, it includes the meaning of computer simulation, which means simulating or testing a target phenomenon on a computer, and analyzing and measuring the phenomena that occur in a virtual space constructed on the computer. Specifically, the simulation involves performing calculations or measurements under constantly changing environmental conditions, and accumulating the calculated or measured results each time.

[0023] The subject of the information processing device 10 can experience driving these means of transportation in a virtual space, and this experience can be simulated (measured and stored in real time) from an overhead perspective.

[0024] In Interactive Multi-Simulator 5, Avatar 6 and Model 7 interact. This model is an interpersonal-responsive individual behavior model. Human behavior changes in response to the actions of others. In other words, people perceive the actions of others, decide what action to take, and then actually act. A predictive model is created that predicts human behavior based on this sequence of perception → judgment → action, and an integrated mobility simulation 11 is performed using this predictive model to predict traffic and danger in a virtual city. The predictive model is updated by feeding back the prediction and evaluation results from this integrated mobility simulation 11. A network traffic model 12 is also created in this integrated mobility simulation 11. This network traffic model 12 is linked with the interpersonal-responsive individual behavior model. The prediction and evaluation results from this integrated mobility simulation 11 may include acceptability evaluation, accident risk evaluation, congestion prediction, urban planning, etc.

[0025] Based on the prediction and evaluation results from the integrated mobility simulation 11 described above, policies and plans are improved in the physical space 14, traffic and pedestrian flow are measured, mobile big data 16 is created, and this real-world data 15 is materialized in cyberspace 13, where simulations are performed by the Interactive Multi-Simulator 5. This cycle is repeated.

[0026] Referring to Figure 2, the positioning of Digital Inclusive Mobility 20, Simulation 17, Digital Twin 18, and Metaverse 19 will be explained. Figure 2 is a diagram illustrating the positioning of Digital Inclusive Mobility 20, Simulation 17, Digital Twin 18, and Metaverse 19 according to this embodiment.

[0027] Digital Inclusive Mobility 20 performs computations by moving between and merging cyberspace (virtual space) 13 and physical space (real space) 14. Digital Inclusive Mobility 20 also performs computations by moving between and merging digital twin 18 and metaverse 19. A link between virtual and real is essential for digital twin 18, while a link between virtual and real is not essential for metaverse 19. In other words, data is acquired in real time from physical space (real space) 14, and the real space is reproduced bidirectionally from cyberspace 13. The subject participates in the metaverse 19 as avatar 6, and the simulation is performed in cyberspace 13. Avatar 6 includes not only the real appearance but also a future image. In the simulation, predictions are made using real and virtual data as input in a space that is not necessarily on a PC. Digital Inclusive Mobility 20 functions as an information processing device 10.

[0028] Next, with reference to Figure 3, the usage of the information processing device 10 according to this embodiment will be described. Figure 3 is a diagram illustrating the usage of the information processing device 10. Since human behavior changes in response to the behavior of others, interaction representation is necessary. Data is necessary to build a model, but it is impossible to measure data that can be observed and experienced from a bird's-eye view in real space with reproducibility. Therefore, a measurement environment is needed in cyberspace 13 where multiple people can mutually recognize each other using different means of transportation (avatars 6).

[0029] The various simulators shown in Figure 3 refer to the VR+walking device 21, the driving simulator 22, and the cyclist simulator 23, but are not limited to these; simulators used by mobile devices as traffic participants, such as electric scooters, are also included.

[0030] The left side of Figure 3 is a diagram illustrating the VR+walking device 21 according to this embodiment. The VR+walking device 21 is a device that simulates the behavior and reactions of a pedestrian 3 under various road conditions and walking conditions, and can simulate walking, allow for safe learning of walking techniques, and provide a simulated experience to other road users. The VR+walking device 21 comprises a right controller 21a, a left controller 21b, and a head-mounted display (HMD) 21c, all equipped on the pedestrian 3. The HMD 21c displays images of the virtual space to the pedestrian. The right controller 21a and the left controller 21b are operated by holding them in the left and right hands, respectively, as shown in Figure 3. The pedestrian walks by swinging both arms, which hold the right controller 21a and the left controller 21b, back and forth, and can determine the direction of movement in the virtual space by changing the magnitude of the arm swing on the left and right sides. The magnitude of the arm swing (stroke amount) is converted into a movement amount, and the positions of the right controller 21a and the left controller 21b are averaged to determine the direction of travel. When stopped, both arms holding the right controller 21a and the left controller 21b should be lowered.

[0031] Pedestrian 3 can display a posture in the virtual space where the pedestrian object (avatar 6) raises one arm by raising either the right controller 21a or the left controller 21b to a position higher than the height of the HMD 21c.

[0032] The central part of Figure 3 is a diagram illustrating the driving simulator 22 according to this embodiment. The driving simulator 22 is a device for a subject to experience driving a simulated vehicle 22a. The driving simulator 22 is a device that simulates the behavior of a vehicle 1 and the driver's reactions under various road conditions and driving conditions, and can simulate actual driving, allow for safe learning of driving skills, test the performance and behavior of the vehicle 1, and provide a simulated experience to other road participants. A subject is a person who participates in cyberspace 13 using the information processing device 10 to experience simulated driving of a vehicle. A simulated vehicle 22a is a device that simulates how an automobile behaves while driving under various road conditions and driving conditions, or how a driver reacts while driving.

[0033] The basic configuration of the simulated vehicle 22a consists of a field of view simulation device, an acoustic simulation device, a steering sensation simulation device, a kinesthetic sensation simulation device, a control unit, etc. However, not all of these are essential components, and the essential components differ depending on the purpose of use of the simulated vehicle.

[0034] The field of view simulation device displays the scenery outside the vehicle (road, background, obstacles, preceding vehicles, oncoming vehicles, etc.) as seen by the driver through the window. In this embodiment, it is configured with several liquid crystal displays as a set. The field of view image is updated approximately every 50 to 100 ms based on vehicle motion information.

[0035] The sound simulation device uses data from actual vehicles, such as engine noise, tire noise, and wind noise, to simulate the noise levels of a real vehicle according to the driving conditions, giving the driver a sense of speed.

[0036] The steering feel simulation device simulates steering wheel resistance, instrument displays, and movements. It uses a motor to provide steering feel and simulates speed through the movement of the meters. It also reproduces the force and resistance of the accelerator and brake pedals to closely resemble those of a real vehicle.

[0037] A kinesthetic simulation device is a device that simulates inertial forces and vibrations from vehicle motion. By moving the driver's seat where the driver sits and the field of view simulation device, the acceleration of the vehicle is simulated through these movements.

[0038] The control unit calculates the vehicle's motion based on the driver's input (amount of pressure applied to the accelerator and brake pedals, steering input, gear selection, etc.).

[0039] The right side of Figure 3 is a diagram illustrating the cyclist simulator 23, which is a bicycle driving simulation device according to this embodiment. The cyclist simulator 23 is a device that simulates the behavior of a bicycle 2 and the reactions of the driver under various road conditions and driving conditions. It is a device for subjects to experience driving a simulated bicycle 23a, which can simulate actual driving, allow them to learn driving techniques safely, test the performance and behavior of the bicycle 2, and provide a simulated experience to other road users. A subject is a person who participates in cyberspace 13 using the information processing device 10 to experience simulated bicycle driving. A simulated bicycle 23a is a device that simulates how a bicycle behaves while driving under various road conditions and driving conditions, or how a driver reacts while driving.

[0040] Refer to Figure 4 to see an example in which the information processing device 10 is implemented on the server 31 of a client-server system. Figure 4 is a diagram showing an example in which the information processing device 10 is implemented on the server 31 of a client-server system.

[0041] The information processing device 10 according to this embodiment is implemented as the server 31 of a client-server system. The bicycle simulator 32, vehicle simulator 33, vehicle simulator 34, pedestrian simulator 35, and other traffic agents 36 are implemented as clients of the client-server system.

[0042] (Functional configuration of the information processing device 10) Next, with reference to Figure 5, an example of the functional configuration of the information processing device 10 will be described. Figure 5 is a block diagram illustrating an example of the functional configuration of the information processing device 10.

[0043] The information processing device 10 includes a measurement unit 40, a storage unit 41, a model generation unit 42, a simulation execution unit 43, a collection unit 44, a storage unit 45, and an analysis unit 46.

[0044] The measurement unit 40 measures the movement simulations of multiple users in real space and reflects the measured movement simulation results in the virtual space. The multiple users may be, for example, people (pedestrians), cyclists, car drivers, or users of new mobility devices. For example, there may be one person (pedestrian), five people, or ten people; for example, there may be one person, five people, or ten people; for example, there may be one car driver, five people, or ten people; and for example, there may be one person, five people, or ten people.

[0045] The memory unit 41 stores the movement status of multiple users. These multiple users may be, for example, cyclists, car drivers, or users of new mobility devices, and the movement status may include, for example, stable driving, reckless driving including driving against traffic or aggressive driving. The memory unit 41 is implemented using an HDD (Hard Disk Drive), SSD (Solid State Drive), online storage, etc., and stores the OS, information processing programs described later, other application software, and various data used by these programs. The memory unit 41 also stores various data acquired or generated by the information processing device 10.

[0046] The model generation unit 42 generates simulation model data based on movement conditions and stored data. Model data is data generated based on the driving conditions of the driver of car 1, who participated as an avatar, and stored data of pedestrians 3. Specifically, for example, this data might include information such as drivers who drive recklessly exceeding the speed limit or pedestrians who walk quickly running red lights. The model generation unit 42 reproduces the traffic situation in cyberspace 13 based on parameters including the ratio of pedestrians to vehicles and the ratio of different user personalities and physical characteristics. The ratio of pedestrians to vehicles is, for example, 50% pedestrians and 50% vehicles. Parameters are indicators for generating model data, and specifically include, for example, the user's personality and physical characteristics. Examples of user personalities include aggressive and conservative. Examples of physical characteristics include height (eye level) and weight.

[0047] The simulation execution unit 43 performs a simulation based on model data reflected in the virtual space when multiple users participate in cyberspace 13 using different means of transportation. Different means of transportation include, for example, bicycles, automobiles, and new mobility devices, and the simulation includes, for example, road hazard levels and congestion levels.

[0048] The data collection unit 44 collects the results of the simulation performed by the simulation execution unit 43. The simulation results may include, for example, a risk level of 60% or a congestion level of 80%.

[0049] The storage unit 45 stores the simulation results collected by the collection unit 44.

[0050] The analysis unit 46 analyzes the simulation results stored by the storage unit 45. The results of the analysis are used, for example, for children's experiential learning, traffic safety instruction, and to visualize the digital transformation of cities.

[0051] Furthermore, these are functional units realized by the CPU executing information processing programs deployed on RAM. The information processing device 10 also appropriately implements storage units such as ROM, RAM, SSD, HDD, and various interfaces (not shown).

[0052] Figure 6 is a diagram illustrating a participatory and experiential simulator in a space where pedestrians and vehicles coexist, as described in the information processing device 10 according to this embodiment. Here, multiple people can simultaneously share a virtual travel space. The road shown in the center of Figure 6 can be simultaneously used by people using a driving simulator, a cycling simulator, and a walking simulator.

[0053] The upper left of Figure 6 shows the vehicle view 61 as seen from the perspective of the person using the driving simulator. The vehicle view 61 displays the view ahead from the perspective of a vehicle traveling on the left side of the road. In this case, the view is as if traveling at 30 km / h.

[0054] The lower left of Figure 6 shows the bicycle view 62 as seen from the perspective of a user of the cycling simulator. The bicycle view 62 displays the scenery ahead as seen from the perspective of a cyclist riding on the sidewalk on the right.

[0055] The lower right of Figure 6 shows the pedestrian viewpoint 63 as seen from the perspective of a person using the walking simulator. The pedestrian viewpoint displays the scenery in front of a pedestrian attempting to cross a crosswalk.

[0056] (Regarding information processing methods and information processing programs) Next, with reference to Figure 7, an information processing program according to one embodiment of the present invention will be described along with an information processing method. Figure 7 is an example of a flowchart of the information processing program according to this embodiment. The information processing method is executed by the processing unit of the information processing device 10 based on the information processing program.

[0057] The information processing program includes a measurement step S10, a storage step S20, a model generation step S30, a simulation execution step S40, a data collection step S50, a storage step S60, and an analysis step S70, among others.

[0058] The information processing program enables the processing unit of the information processing device 10 to perform functions such as measurement, storage, model generation, simulation execution, data collection, storage, and analysis. These functions are executed in the order shown in the flowchart of Figure 7, but the order can be changed as appropriate. Since each function overlaps with the descriptions of the various functional units of the information processing device 10 mentioned above, detailed explanations are omitted.

[0059] The measurement function measures the driving simulations of multiple users in the real world and reflects the measured driving simulation results in the virtual world (S10: measurement step).

[0060] The memory function stores the driving status of multiple users (S20: memory step).

[0061] The model generation function generates simulation model data based on driving conditions and stored data, and reproduces traffic conditions in a virtual space based on parameters including the proportion of pedestrians and vehicles mixed together, and the proportion of users with different personalities and physical characteristics mixed together. (S30: Model generation step).

[0062] The simulation execution function runs a simulation based on model data reflected in the virtual space when multiple users participate in the virtual space using different means of transportation (S40: Simulation execution step).

[0063] The data collection function collects the results of simulations performed by the simulation execution function (S50: data collection step).

[0064] The storage function stores the simulation results collected by the data collection function (S60: storage step).

[0065] The analysis function analyzes the simulation results accumulated by the data storage function, and the results of the analysis are used for children's experiential learning, traffic safety instruction, and visualizations of the digital transformation of the city (S70: Analysis step).

[0066] According to this embodiment, it is possible to provide a diverse virtual space participation mobility environment involving multiple entities, taking into account the prediction of human behavior in relationships with others, i.e., interactions.

[0067] Furthermore, the present invention is not limited to the information processing apparatus 10, information processing method, and information processing program according to the above-described embodiment, and can be implemented by various other modifications or applications without departing from the gist of the present invention as described in the claims. Also, although the word "data" is used in the above-described embodiment, the word "data" can be replaced with "information," and the word "information" can be replaced with "data." [Explanation of Symbols]

[0068] 1 car 2 Bicycles 3 Pedestrians 4. New Mobility 5 Interactive Multi-Simulator 6 Avatars 7 models 8. Interactions 9. The Other-Reactive Individual Behavior Model 10 Information Processing Devices 11. Integrated Movement Simulation 12 Network Transportation Models 13 Cyberspace 14. Physical Space 15 Real-space data 16 Mobile Big Data 17 Simulation 18 Digital Twin 19 Metaverse 20 Digital Inclusive Mobility 21 VR + Walking Device 21a Right controller 21b Left Controller 21c HMD 22 Driving Simulators 22a Simulated Vehicle 23 Cyclist Simulator 23a Simulated bicycle 30 System Configurations 31 Servers 32 Bicycle Simulators 33 Vehicle Simulator 34 Vehicle Simulators 35 Pedestrian Simulator 36 Other Transportation Agents 40 Measuring part 41 Storage section 42 Model Generation Unit 43 Simulation Execution Unit 44 Collection Department 45 Storage section 46 Analysis Department 61 Vehicle View 62 Bicycle Perspective 63 Pedestrian Perspective

Claims

1. A measurement unit that measures the movement simulations of multiple users in real space and reflects the measured movement simulation results in a virtual space, A storage unit that stores the movement status of the aforementioned multiple users, A model generation unit generates simulation model data based on the aforementioned movement status and stored data, When the multiple users participate in the virtual space using different means of transportation, a simulation execution unit executes a simulation based on the model data reflected in the virtual space, Equipped with An information processing device characterized by the following:

2. The information processing device according to claim 1, characterized in that the aforementioned multiple users can participate in the virtual space using multiple types of means of transportation, including vehicles, pedestrians, and cyclists.

3. The information processing device according to claim 1, characterized in that the model generation unit reproduces the traffic situation in the virtual space based on parameters including the proportion of pedestrians and vehicles mixed together and the proportion of users with different personalities and physical characteristics mixed together.

4. A collection unit for collecting the results of the aforementioned simulation, A storage unit that stores the results collected by the collection unit, The aforementioned storage unit analyzes the results that have been stored, Furthermore, it is equipped with The information processing apparatus according to feature 1.

5. On the computer, A measurement step that measures the movement simulations of multiple users and reflects the measured movement simulation results in a virtual space, A storage step for storing the movement status of the aforementioned multiple users, A model generation step that generates simulation model data based on the aforementioned movement status and stored data, A simulation execution step in which, when the multiple users participate in the virtual space using different means of transportation, a simulation is performed based on the model data reflected in the virtual space, Make it run An information processing method characterized by the following:

6. On the computer, A measurement function that measures the movement simulations of multiple users and reflects the measured movement simulation results in a virtual space, A memory function that stores the movement status of the aforementioned multiple users, A model generation function that generates simulation model data based on the aforementioned movement status and stored data, When the multiple users participate in the virtual space using different means of transportation, a simulation execution function is provided that executes a simulation based on the model data reflected in the virtual space. Make it run An information processing program characterized by the following features.

Images