Information processing device, method, program

Abstract

To promote space development, we provide an information processing device, method, and program that improve the appearance of the simulation environment for spacecraft and the accuracy of the simulation of the movement of various objects. [Solution] The method involves providing a virtual space in which a first object representing the interior space of a spacecraft is rendered by a rendering engine, providing a physics simulator that performs a physical simulation with information that controls the behavior of a second object different from the first object, which is movably positioned inside the first object, thereby causing the physics simulator to simulate physical phenomena affecting at least one of the first object and the second object, and generating parameters to be given to the rendering engine by performing an abstraction process on the simulation results from the physics simulator.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus, method, and program.

Background Art

[0002] During the development of products and the simulation of systems during operation, simulations are being carried out. Objects are placed in a virtual space, and simulations are performed through physical calculations. By acquiring information from the real space and reproducing the real space environment in the virtual space, for example, discussions are held on improvement points of factories and production lines, and the process is considered, which is sometimes referred to as a digital twin.

[0003] In Patent Document 1, as for performing physical simulation of a game, it is described that "applications such as video games operating on a computer system may require both physical simulation and graphics rendering", and "using a graphics processor to perform physical simulation of a game".

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Meanwhile, as space development progresses, it is expected that private companies will develop rockets and launch spacecraft in addition to national projects. As a simulation environment for space development, for example, by providing a digital twin of a spacecraft in space, technological development by companies developing spacecraft, onboard equipment, and robots that operate inside the spacecraft will be encouraged, and entry into the space industry will be further promoted. Therefore, if the visual reproduction of a real spacecraft is highly accurate in the digital twin, it will become easier for the companies developing it to carry out development.

[0006] On the other hand, in outer space, various activities within a spacecraft are performed under constraints different from those on Earth, such as gravity. Operating a spacecraft in outer space is also more difficult than on Earth, including repairs and the transportation of necessary materials. Therefore, in order to ensure stable operation in outer space and to prevent unexpected malfunctions as much as possible, it is important to perform high-resolution physical simulations that simulate actual operation in outer space, for example, when there are moving objects inside the spacecraft.

[0007] As described above, in order to further promote space development, there is a need for technology that can accurately reproduce spacecraft, including their interiors, in a simulation environment, while also performing high-precision physical simulations of the movement of various objects. [Means for solving the problem]

[0008] According to one embodiment, a program is provided for operating a computer having one or more computer processors. The program causes one or more computer processors to perform the following steps: provide a virtual space to which a first object representing the interior space of a spacecraft is rendered by a rendering engine; provide a physical simulator that performs physical simulations with information that controls the movement of a second object different from the first object, which is movably positioned inside the first object, thereby causing the physical simulator to perform a simulation of physical phenomena affecting at least one of the first object and the second object; generate parameters to be given to the rendering engine by performing an abstraction process on the simulation results from the physical simulator; and perform a physical simulation in the virtual space associated with the movement of the second object using the generated parameters and the physical simulation functions provided by the rendering engine. [Effects of the Invention]

[0009] According to this disclosure, we can provide technology that can further improve the appearance of the environment in spacecraft simulations and the accuracy of the simulation of the behavior of various objects, thereby encouraging entry into the space industry and further promoting space development. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows the configuration of System 1. [Figure 2] Figure 2 shows the configuration of server 20. [Figure 3] Figure 3 shows the configuration of terminal 10. [Figure 4] Figure 4 shows the data structure of the user database 211. [Figure 5] Figure 5 shows the data structure of the spacecraft database 212. [Figure 6] Figure 6 shows the data structure of the robot database 213. [Figure 7] Figure 7 shows the data structure of the constraint database 214. [Figure 8] Figure 8 shows the data structure of the command content database 215. [Figure 9] Figure 9 shows the data structure of the physical simulation results database 216. [Figure 10] Figure 10 shows the process flow in which a computational fluid dynamics simulation is performed using a physical simulator, and then parameters to be applied to the rendering engine are generated from the simulation results, and rendering is performed. [Figure 11] Figure 11 shows an example of an operation screen that displays the results of a physical simulation performed according to environmental conditions in outer space. [Modes for carrying out the invention]

[0011] Embodiments of this disclosure will be described below with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated. The following embodiments are not intended to unduly limit the content of this disclosure as described in the claims. Not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each figure is a schematic diagram and is not necessarily a strict illustration.

[0012] Furthermore, in the following description, "processor" refers to one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

[0013] Further, at least one processor may be a processor in a broad sense, such as a hardware circuit (e.g., FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0014] In the following description, an expression such as "xxx table" may be used to describe information from which an output is obtained for an input. However, this information may be data of any structure or a learning model such as a neural network that generates an output for an input. Therefore, "xxx table" can be referred to as "xxx information".

[0015] In the following description, the configuration of each table is an example. One table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0016] In the following description, the "program" may be used as the subject to describe processing. However, since the program is executed by a processor to perform defined processing while appropriately using a storage unit and / or an interface unit, etc., the subject of the processing may be the processor (or a device such as a controller having the processor).

[0017] The program may be installed in a device such as a computer, or may be in, for example, a program distribution server or a computer-readable (e.g., non-temporary) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0018] In the following description, an identification number is used as identification information for various objects, but identification information of other types (e.g., an identifier including letters and symbols) may be adopted.

[0019] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0020] Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0021] <Outline of Embodiment 1> To promote space development and encourage the entry of space development companies, it is conceivable to provide a simulation environment that replicates spacecraft in outer space. By enabling the simulation of the spacecraft itself, the equipment installed on it, and the movements of robots that move and work inside or outside the spacecraft, under conditions such as gravity corresponding to the spacecraft's orbit, it is believed that the development of these will become even easier.

[0022] In such a simulation environment, the appearance of a real spacecraft can be reproduced in high definition, providing a sense of being inside the spacecraft. Furthermore, by faithfully reproducing the interior and exterior of the spacecraft, the accuracy of collision detection with interior walls, equipment, etc., can be improved, for example, when a robot working inside the spacecraft moves around and performs tasks, taking into account the effects of gravity. In this way, this embodiment provides a simulation environment with improved rendering accuracy, thereby encouraging various businesses to participate in space development.

[0023] On the other hand, while high-resolution rendering engines can move objects using physics calculations, the accuracy of those calculations is limited to what the rendering engine provides. When a spacecraft is operating in space, the difficulty of its maintenance and operation is significantly higher than when it is on Earth. Therefore, when simulating the movements of spacecraft, their equipment, and robots operating within the spacecraft, it is necessary to improve the accuracy of the simulation of these movements themselves (for example, the effect of a space robot's movement within the spacecraft on airflow). In addition, changes in the air pressure distribution and airflow within the spacecraft due to the movement of robots within the spacecraft may result in loads on the equipment inside the spacecraft, so it may be necessary to verify what kind of equipment is affected and to what extent.

[0024] Therefore, by increasing the rendering resolution and improving the accuracy of simulations as described above, it is believed that the discovery of challenges in actual spacecraft operations will be facilitated, and space development will be further promoted.

[0025] Therefore, in this embodiment, the following will be described as technologies that broaden the base of space development and make it easier to enter the field. (1) This section describes a simulation environment that reflects the conditions of a spacecraft in outer space (gravity according to orbit, the effects of sunlight, etc.). For businesses that are just starting out in space development, it is not easy to set up a simulation environment with conditions (gravity, etc.) that are different from those on Earth. In response to this, the system will make it easy to set conditions such as gravity in the expected orbit by specifying the orbit. This will allow, for example, simulations to be performed in a virtual space that mimics the interior of a spacecraft, taking into account the effects of gravity on the movement of various objects such as robots. This will reduce the burden on businesses that develop various devices used in spacecraft and further encourage their entry into the space industry. (2) This section describes a technology that provides a virtual space that faithfully reproduces a real spacecraft through high-definition rendering, while also having a physics simulator, which provides higher-definition physics simulation calculations compared to the rendering engine, handle physics simulations that exceed the accuracy supported by the rendering engine. This allows operators to use a virtual space equivalent to that of a real spacecraft, including collision detection, and to develop in a highly immersive environment by using a simulation environment that provides rendering of visually equivalent quality to that of a real spacecraft. Specifically, by providing rendering results from a rendering engine and integrating the results of simulations from a physics simulator that can perform higher-definition and more computationally intensive physics simulations compared to the rendering engine, space development becomes even easier. In this way, by utilizing the results of high-definition physics simulations from a physics simulator, it becomes even easier to conduct space development that is expected to operate stably in space and prevent unexpected failures as much as possible. The following describes technologies that broaden the base of space development and make it easier to enter the field.

[0026] <1.1 System Configuration Diagram> System 1, shown in Figure 1, includes a server 20, user terminals 10 and 10A, a terminal 30 for the administrator of the space development project, a terminal 40 for the client of the space development project, a server 95 for artificial intelligence (large-scale language model) services (hereinafter sometimes referred to as "server 95 for large-scale language model services"), and a server 97 for physical simulation services. These devices communicate with each other via a network 80.

[0027] In this embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, the collection of each device can be considered as a single "information processing device," and System 1 may be formed as a collection of multiple devices. The way in which the multiple functions required to realize System 1 according to this embodiment are distributed to one or more hardware can be appropriately determined in view of the processing capacity of each hardware and / or the specifications required for System 1.

[0028] Server 20 provides the following to the user in general terms: • Provides a digital twin that replicates the interior of an actual spacecraft, and offers a simulation environment in which robots can operate while applying conditions of outer space, such as gravity, within the virtual spacecraft. • Providing various modules that control the operation of spacecraft, and software to optimize the operation of these modules. Providing an operating system for these spacecraft to run the various modules on the spacecraft. This service provides simulations for spacecraft developers, adhering to the constraints of space development. It offers a simulation environment for operating spacecraft in space, and also provides control modules for controlling spacecraft movement, as well as system software to optimize these modules. • A service that matches companies that conduct space development with companies that provide financial support for space development, thereby promoting financial support for space projects. • A service that supports the careers of personnel involved in space development by matching them with companies that conduct space development and facilitating transfers between organizations through job changes, etc. • A service that supports transactions between those who request services related to space development and those who fulfill those requests. The following provides a detailed explanation.

[0029] Server 20 matches space development companies with companies that provide financial support for space development, thereby promoting financial assistance for space projects. Server 20 collects various information on space projects and calculates an evaluation value for each project. By providing these evaluation results to companies that provide financial support, it is possible to encourage loans or investments. Furthermore, based on these evaluation results, it can calculate insurance premiums for space projects and present these premiums to companies involved in space projects, accepting applications from them.

[0030] Server 20 may evaluate the space project by assessing the organizational structure required to carry it out, and evaluates the space project from the perspectives of (1) the development resources required for the space project, and (2) the operational resources required for the space project.

[0031] (1) As part of evaluating the development resources involved in the space project, for example, Server 20 evaluates development resources such as the performance of the spacecraft, the system for developing the spacecraft, and the track record of development in order to assess the likelihood that the space project can be evaluated as successful by achieving its initial objectives.

[0032] (2) As part of the evaluation of operational resources for space projects, Server 20 evaluates the resources needed to launch and operate spacecraft. For example, the following accuracy and track record may be evaluated as part of the evaluation of resources needed to operate spacecraft. • By launching spacecraft, monitoring their orbits, and predicting their trajectories, collisions with other spacecraft such as satellites can be prevented during launch. • Monitoring whether satellites and other spacecraft in orbit are still in orbit. This can be done by monitoring devices on the ground, or by satellites in space sensing other spacecraft. • Controlling the attitude of a spacecraft. For example, to improve the efficiency of generating energy for the spacecraft's operation using solar power, the attitude of the spacecraft may be controlled so that its solar panels are at an optimal angle to the sun. Also, to enable communication with other spacecraft, the attitude of the spacecraft may be controlled so that its antenna is pointed in the appropriate direction. • Controlling the attitude of a spacecraft and controlling thrusters to accelerate or decelerate it. To control such thrusters, a model is prepared that predicts the spacecraft's inertia and its flight path based on thruster control. For example, in order to dock spacecraft with a space station, spacecraft may need to be brought into space at a predetermined position and angle. • Monitoring space objects such as debris and predicting their orbits. For example, if there is a risk of debris colliding with a spacecraft such as a satellite, it may be possible to remove the debris or change the orbit of the spacecraft at risk of collision. As described above, Server 20 can evaluate the likelihood of a space project achieving its intended objectives and other successes by assessing the development and operational resources involved in the space project.

[0033] Here, the server 20 may determine the evaluation value as follows in order to assess the probability of success. • The position of the team involved in a project within the overall picture. For example, whether their development and operational track record is among the top tier (a certain percentage from the top). For example, by aggregating the number of projects they were involved in and the budget size of each project for each company, it is possible to determine the ranking of their development track record. • Whether or not it meets the standards set by the government or other relevant bodies. For example, whether or not the quality of the software code meets certain standards. • The likelihood of success for the current space project compared to past successful or unsuccessful space projects. For example, if the performance of the spacecraft in this project is significantly improved compared to past successful space projects, or if operational experience has been accumulated, the project may be evaluated as having a higher probability of success than past projects. Server 20 provides a service for simulating the space environment. For example, Server 20 receives data related to the design of a ground-based vehicle from the user's terminal 10, determines whether it conforms to constraints such as gravity, temperature, communication, power, air, and radiation in space development, and provides the simulation service by responding with the determination result to the user's terminal 10. Server 20 can also provide the service by having Server 95, an artificial intelligence (large-scale language model) service, generate the content of the response to the user, and then responding with the generated result to the user.

[0034] Server 20 provides a service to device developers that performs simulations according to the constraints of space development. For example, it constructs the environment inside a spacecraft (ISS: International Space Station, space colony), the environment outside the spacecraft, the environment in orbit, the lunar surface environment such as terrain and day / night cycle, and the surface environment of planets such as Mars in a virtual space, and performs simulations of controlling spacecraft objects placed in that virtual space.

[0035] Server 20 provides services to support career changes and other transitions for personnel involved in space development. For example, Server 20 evaluates personnel involved in space development based on their track record of participating in space projects. In this evaluation, Server 20 assesses the compensation (e.g., annual salary) that should be paid for the work performed by personnel involved in space development. Server 20 accepts registration of job information from recruiting companies and matches it with job seekers. Server 20 also presents recruiting companies with the expected compensation for the personnel they are recruiting. This enables the transfer of personnel between organizations.

[0036] Server 20 accepts user registrations and project registrations from clients and administrators of space development projects. Through this, Server 20 provides services to match space development projects with device development companies, including searching for device development companies and searching for space development projects.

[0037] Server 20 is connected to the servers of product manufacturers that provide ready-made products, and accepts product registrations from manufacturers of components, finished products, etc. This allows Server 20 to provide a search function for ready-made products to stakeholders in space development projects and businesses developing devices.

[0038] Server 20 is connected to an advertising distribution service server and delivers advertisements to various users of Server 20, such as device developers, based on their usage history of the services provided by Server 20. For example, based on the history of a device developer designing a spacecraft using Server 20's simulation service, advertisements can be delivered that are tailored to the purpose of the spacecraft and the components that make up the spacecraft, thereby promoting matching with products offered by advertisers.

[0039] Terminal 10 is the terminal of a user who uses the services provided by Server 20. In the illustrated example, terminals 10, terminal 10A, etc., are shown as terminals used by users of the services provided by Server 20, but each user operates their own terminal. For example, a user of a space project development company, a user of an investment company, and a user of a lending company each operate Terminal 10.

[0040] Server 95 of the Large-Scale Language Model Service is a server that executes language processing tasks using language models built through learning processes including artificial intelligence (AI). An LLM (Large Language Model) is a model that has been pre-trained on large amounts of data (such as text data), for example, a large amount of web content on the internet, or a large amount of data stored in a designated database, and can perform various language processing tasks by being given a task.

[0041] The server 95 of the large-scale language model service accepts prompt input in the form of text, images, audio, etc., and generates and responds with answers to those prompts. Examples of LLMs include GPT-3 and GPT-4 developed by OpenAI, and BERT developed by Google.

[0042] The physical simulation service server 97 is a server that provides high-resolution physical simulation capabilities, for example, by providing physical simulations using computational fluid dynamics.

[0043] The spacecraft being simulated is a space object flying through space. Based on sensing results obtained by the spacecraft, the spacecraft controls each module to perform control according to the spacecraft's situation in space (e.g., temperature adjustment according to the temperature environment, attitude control according to cosmic radiation, attitude / orbit control to avoid collisions with other space objects, etc.), and control to continue operating the spacecraft even if an anomaly occurs in space (e.g., energy control in the event of a power generation device failure, fuel control for thruster control, water generation and replenishment according to the amount of water and food consumed by the human body, setting the importance and schedule of communications according to the communication capacity, etc.). In this embodiment, the spacecraft controls each module to operate automatically without manual operation by an operator. For example, the spacecraft sets a schedule and priority for controlling the operation of various modules of the spacecraft based on monitoring results including the spacecraft's own sensing results in space, and information provided by a ground station, and operates the various modules according to these.

[0044] Ground stations monitor the orbits of space objects (spacecraft, satellites, debris, etc.) in outer space, predict their trajectories, send commands to spacecraft to avoid collisions, and send commands to spacecraft to control their modules according to their trajectories.

[0045] In the simulation environment provided by Server 20, the conditions in the simulation environment (such as gravity) may be set to control the spacecraft to avoid collisions as described above, according to the orbit of the spacecraft being simulated, and the movements of robots, etc., may be simulated.

[0046] The configuration of each device is described below.

[0047] The server 20 includes a communication interface 22, an input / output interface 23, memory 25, storage 26, and a processor 29.

[0048] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0049] Input / Output IF23 functions as an interface between an input device for receiving user input operations and an output device for presenting information to the user.

[0050] Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0051] Storage 26 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0052] The processor 29 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0053] Terminal 10 can be implemented, for example, as follows: • Handheld mobile devices such as smartphones and tablets • Stationary PCs (Personal Computers), Laptop PCs • Wearable devices worn by the user (watch-type, glasses-type, etc.) Terminal 10 includes a communication interface (IF) 12, an input device 13, an output device 14, memory 15, storage 16, and a processor 19.

[0054] The communication interface 12 is an interface for inputting and outputting signals so that terminal 10 can communicate with an external device.

[0055] The input device 13 is a device for receiving input operations from the user (for example, a touch panel, touchpad, pointing device such as a mouse, keyboard, etc.).

[0056] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0057] Memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0058] Storage 16 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0059] The processor 19 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0060] <1.2 Functional Configuration of Server 20> Figure 2 shows the configuration of server 20. As shown in Figure 2, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0061] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0062] The memory unit 202 stores various databases, such as the user database 211, the spacecraft database 212, the robot database 213, the constraints database 214, the command content database 215, and the physical simulation results database 216.

[0063] User Database 211 is a database that manages information on each user involved in the development of spacecraft. Further details will be provided later.

[0064] Spacecraft Database 212 is a database that manages information on 3D models of space objects flying through space that are the subject of simulations. Further details will be provided later.

[0065] Robot Database 213 is a database that manages information on 3D models of space robots that operate inside and outside spacecraft in outer space, which are the subjects of simulations. Further details will be provided later.

[0066] Constraint Database 214 is a database that holds constraints in space development. Further details will be provided later.

[0067] The command content database 215 is a database that manages the content of commands issued to spacecraft or space robots in the simulation environment. Further details will be described later.

[0068] The physical simulation results database 216 is a database that manages the simulation results from the physical simulation service server 97 and the physical calculations performed by the rendering engine based on those results. Further details will be described later.

[0069] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs the functions shown as the receive control module 2041, the transmit control module 2042, the user management module 2043, the spacecraft / space robot registration module 2044, the simulation processing module 2045, the rendering processing module 2046, and the physical simulator control module 2047.

[0070] The receive control module 2041 controls the process by which the server 20 receives signals from external devices according to a communication protocol.

[0071] The transmission control module 2042 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0072] The user management module 2043 is a module for managing information for each user using System 1. Specifically, the user management module 2043 accepts registration of each user's information and updates the user database 211.

[0073] The Spacecraft / Space Robot Registration Module 2044 is a program module that accepts registrations of information on spacecraft and space robots (such as 3D model information) that are the subject of simulations, and updates the Spacecraft Database 212, Robot Database 213, etc.

[0074] Simulation processing module 2045 is a program module that provides spacecraft developers with the functionality to simulate the space environment.

[0075] The simulation processing module 2045 provides users with space environment simulation capabilities in the following manner: • Constructs a three-dimensional virtual space that reflects the conditions inside and outside the ISS, and on the lunar or planetary surface (gravity, atmosphere, etc.), and returns simulation results of controlling a spacecraft (in-space drone, satellite, lunar rover, etc.). The system accepts registration of device design data from users and determines whether the device meets the constraints based on information about constraints such as gravity, as shown in the constraint database 214. For example, upon receiving registration of ground-based device design data, if it does not meet the communication constraints in space development, the system determines that it cannot be used in the space environment (will not operate as intended) and presents the determination result to the user. Here, design data may be information from the device specifications, or information such as device design drawings or 3D models. The rendering processing module 2046 is a program module that provides a virtual space that reproduces the interior of a spacecraft, etc., by performing rendering based on 3D model information managed in the spacecraft database 212, robot database 213, etc.

[0076] The rendering module 2046 renders the interior and exterior of the spaceship according to the virtual camera settings, based on information about the spaceship object and the space robot object, and displays the rendering results on terminal 10. In addition, the rendering module 2046 uses the physics calculation function of the rendering engine to perform physics calculations according to the settings for gravity and other factors in outer space, thereby moving objects and performing collision detection.

[0077] Furthermore, the rendering processing module 2046 receives commands for objects such as spacecraft and space robots (either specified by the user or not), and updates the command content database 215 by generating parameters to drive each object according to the command (for example, generating parameters to move a space robot in response to a command to move it).

[0078] The physics simulator control module 2047 uses the physics simulation service server 97 to identify the simulation parameters to be provided to the physics simulation service server 97 (parameters for driving objects such as space robots to move in space), and instructs the physics simulation service server 97 to perform a high-resolution simulation compared to the rendering engine, including a simulation using computational fluid dynamics. For example, the 3D space may be divided into multiple blocks at a granularity that is difficult for the rendering engine to process, and the physics simulation may be performed by doing so. Alternatively, if the rendering engine does not support simulations using computational fluid dynamics, the physics simulation service server 97 may be instructed to perform simulations of airflow, etc., using computational fluid dynamics. The physics simulator control module updates the physics simulation result database 216 by performing the simulation in this way. The physics simulator control module 2047 obtains the simulation results from the physics simulation service server 97 and generates parameters for performing physics calculations by the rendering engine by abstracting those simulation results (for example, by reducing the data resolution). The physics simulator control module 2047 then uses the generated parameters to perform physics calculations in the rendering engine. Details will be described later.

[0079] <1.3 Configuration of Terminal 10> Figure 3 shows the configuration of terminal 10.

[0080] As shown in Figure 3, terminal 10 includes multiple antennas (antenna 111, antenna 112), communication units corresponding to each antenna (first communication unit 120, second communication unit 121), an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a storage unit 180, and a control unit 190. Terminal 10 also has functions and configurations not specifically shown in Figure 3 (for example, a battery for maintaining power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in Figure 3, each block included in terminal 10 is electrically connected by a bus or the like.

[0081] Antenna 111 radiates signals emitted by terminal 10 as radio waves. Antenna 111 also receives radio waves from space and provides the received signals to first communication unit 120.

[0082] Antenna 112 radiates signals emitted by terminal 10 as radio waves. Antenna 112 also receives radio waves from space and provides the received signals to the second communication unit 121.

[0083] The first communication unit 120 performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 111 in order to communicate with other wireless devices. The second communication unit 121 also performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 112 in order to communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are a communication module that includes a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first communication unit 120 and the second communication unit 121 perform modulation and demodulation, frequency conversion, etc., of the wireless signals transmitted and received by the terminal 10, and provide the received signal to the control unit 190.

[0084] The input device 130 has a mechanism for receiving user input operations. Specifically, the input device 130 is configured as a touchscreen and includes a touch-sensitive device 131. The touch-sensitive device 131 receives user input operations of the terminal 10. The touch-sensitive device 131 detects the user's contact position with the touch panel, for example, by using a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the user's contact position detected by the touch panel to the control unit 190 as an input operation.

[0085] The display 132 displays data such as images, videos, and text in accordance with the control of the control unit 190. The display 132 is implemented by, for example, an LCD or an organic EL display.

[0086] The audio processing unit 140 modulates and demodulates the audio signal. The audio processing unit 140 modulates the signal received from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is implemented, for example, by an audio processing processor. The microphone 141 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 140. The speaker 142 converts the audio signal received from the audio processing unit 140 into sound and outputs the sound to the outside of the terminal 10.

[0087] The location information sensor 150 is a sensor that detects the location of the terminal 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current location of the terminal 10, which is equipped with a GPS module, is detected based on the received signals.

[0088] Camera 160 is a device that receives light using a photodetector and outputs it as an image. Camera 160 is, for example, a depth camera that can detect the distance from camera 160 to the object being photographed.

[0089] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10.

[0090] The storage unit 180 is composed of, for example, flash memory and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.

[0091] The control unit 190 controls the operation of the terminal 10 by reading the program stored in the memory unit 180 and executing the instructions contained in the program. The control unit 190 is, for example, an application processor. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a memory control unit 195.

[0092] The operation reception unit 191 processes input operations from the user to an input device such as a touch-sensitive device 131. Based on the coordinate information of the touch-sensitive device 131 where the user's finger or the like has made contact, the operation reception unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation.

[0093] The transmitting / receiving unit 192 performs processing to enable the terminal 10 to send and receive data with an external device such as a server 20 in accordance with a communication protocol.

[0094] The data processing unit 193 performs calculations on the data received as input by the terminal 10 according to the program and outputs the calculation results to memory or other locations.

[0095] The notification control unit 194 performs the following processes: displaying the display image on the display 132, outputting sound to the speaker 142, and generating vibrations.

[0096] The memory control unit 195 controls the storage of data to the memory unit 180.

[0097] The various types of information stored by the memory unit 180 will now be explained. In a given scenario, the memory unit 180 stores various types of information, such as user information 181 and ground machine design information 182.

[0098] User information 181 is information about a user who uses the services of server 20.

[0099] Ground equipment design information 182 is information about the design data of the ground equipment. For example, if the user is a company that develops, maintains, and operates ground equipment, it will have design data such as specifications, design drawings, and test data for that ground equipment. <2 Data Structure> Figure 4 shows the data structure of the user database 211. The user database 211 includes the fields "User ID", "Name", "Email Address", "Business ID", "Profile", "Attributes", and "Evaluation Score".

[0100] The "User ID" field is information that identifies each user.

[0101] The "Name" field contains information indicating the user's name.

[0102] The "Email Address" field contains the user's email address information for contact purposes.

[0103] Specifically, the item "email address" contains email address information, which serves as user identification information for accepting user logins to the services provided by server 20.

[0104] The "Business ID" field is information that identifies the organization to which the user belongs.

[0105] Specifically, the "Business ID" field is information that identifies the organization to which the user belongs, and the following types of organizations are possible: • Businesses that design, develop, manufacture, process, and transport devices, software, etc., related to space development. • Client for space development projects • An administrator who manages the progress of a space development project based on a commission from the project's client. Businesses that provide financial support to space development projects through loans or investments. The "Profile" item contains information about the user's profile.

[0106] Specifically, the "Profile" field stores the following information as profile information entered by the user: • Technologies with a proven track record • Specialized fields • Career history including previous employers • Areas in which we have a track record of providing loans. • Areas in which we have a track record of investment The "Attributes" field contains information about the attribute tags assigned to the user.

[0107] Specifically, the "Attributes" field allows users to be associated with tags corresponding to their roles in the project, enabling these tags to be used for user searches and other purposes. • Tags for device developers (tags indicating that technology development is being conducted) • Project Manager Tag • Tags of project clients • Material procuring tag Additionally, tags may be assigned according to the role of providing financial support to the project. • Loan officer tag • Investment Manager Tag The "Evaluation Score" item contains information about the evaluation value associated with the user.

[0108] Specifically, the item "Evaluation Score" may retain information on the score calculated as follows as an evaluation value for the user designing the device. In addition to being a user evaluation, it may also be retained as an evaluation value for the business to which the user belongs. For example, the business's evaluation value may be determined based on the evaluation values ​​of users belonging to the business (averaging, weighting, etc.), or the evaluation of a user may be treated as the evaluation value of the business to which the user belongs. • History of developing spacecraft, equipment installed on spacecraft, and space robots used on spacecraft using the digital twin simulation environment provided by Server 20. • History of designing various modules that operate spacecraft flying in outer space. • History of designing various modules that operate the spacecraft using the simulation service provided by Server 20. • History of device design using the simulation service provided by Server 20 • History of devices designed using the simulation service provided by Server 20 being viewed by other users, and history of their adoption in projects. • In the matching function provided by Server 20, the history of searches by other users and the history of successful matches (transactions that have started) are recorded. • History of developing spacecraft • A track record of personnel belonging to the company participating in space projects. Figure 5 shows the data structure of the spacecraft database 212. The spacecraft database 212 includes the following items: "Spacecraft ID", "Type", "Specifications", "Orbit", "Purpose", "Operation Period", and "3D Model Data".

[0109] The "Spacecraft ID" field is information that identifies each 3D model of a space object that is the subject of the simulation.

[0110] The "Type" field contains information about the type of space object being simulated.

[0111] The "Type" field may include the following information regarding the type of space object being simulated: • Spacecraft (space stations capable of housing living organisms such as humans, and those that fly through space for space travel, etc.) ·debris ·Artificial satellite • Other spacecraft equipped with various devices (for example, those launched and flying through space for purposes such as photographing outer space) The "Specifications" item contains information about the specifications of the space object being simulated, such as its size and function.

[0112] The item "Specifications" may include the following as information about the specifications of the space object being simulated: • Indicators indicating the size and weight of a space object, such as mass (total weight at launch, total weight in space, etc.) and dimensions (overall size, shape). Specifications of various devices installed on the spacecraft, such as: specifications of the propulsion system for attitude control (e.g., propellant used to move the spacecraft, engine type), specifications of the power supply system (e.g., solar cells, batteries), communication system (e.g., method of communication with the ground, frequency band), thermal control system (system for managing heat, such as heat sinks, heaters), equipment for achieving the mission of the space project (onboard observation instruments, experimental equipment, etc.), and durability (resistance to radiation and temperature changes in the space environment). • Capacity, etc., the number of people who can use a spacecraft when living organisms such as humans are using it. The "Orbit" item contains information about the orbit of the space object being simulated.

[0113] The item "Orbit" may include the following as information about the orbit of the space object being simulated: · Epoch • Kepler's orbital elements (mean motion, eccentricity, orbital inclination, right ascension of the ascending node, argument of perigee, mean angle of annihilation) • Setting of orbital prediction errors (prediction error for the direction of travel of the spacecraft, prediction error for the perpendicular direction). For example, the server 20 may obtain the results of the calculation of the orbital prediction error of the space object at the ground station. These factors allow us to determine the size of the orbit, the degree of deviation from a circle, the angle between the spacecraft's orbital plane and the Earth's equatorial plane, the angle difference between the perigee and the ascending node (determining the orbit's position), and the angle between the ascending node and the reference direction (orbit's direction), thereby specifying the shape and arrangement of the orbit.

[0114] For example, the orbital altitude of a spacecraft refers to the distance from the Earth's surface to the spacecraft, and is classified as follows: Low Earth Orbit (LEO): An altitude range of approximately 160 km to 2,000 km. Often used by communication satellites, Earth observation satellites, and other similar devices. Medium Earth Orbit (MEO): An orbit ranging from approximately 2,000 km to 35,786 km in altitude. GPS satellites and other satellites are positioned in this orbit. • Geostationary orbit (GEO): Altitude 35,786 km. The satellite is synchronized with the Earth's rotation and always remains above the same point in the sky. The "Purpose" item contains information about the purpose of the space object being simulated.

[0115] The item "Purpose" may include the following as information about the purpose of the space object being simulated: • Observe designated targets (such as meteorological phenomena) • Space stations where people stay • Items used for space travel • For creating video and other content for advertising, etc. (e.g., equipped with a camera or recording device) The item "Operational Period" contains information about the operational period settings for the spacecraft being simulated.

[0116] The item "Operating Period" may include the following information as part of the setting of the operating period: From the timing of planned launch into orbit to the timing of termination of operations • Space travel and other activities involving flying in outer space for a certain period of time. The item "3D Model Data" contains information about the data files for the 3D model data of the space object being simulated.

[0117] More specifically, the item "3D model data" refers to, for example, 3D model data that replicates a space station.

[0118] Figure 6 shows the data structure of the robot database 213. The robot database 213 includes the following items: "Space Robot ID", "Movement Type", "Purpose", "Specifications", "3D Model Data", "3D Model Creator", "3D Model Registration Date", and "3D Model Usage Conditions".

[0119] The item "Space Robot ID" is information that identifies the space robot being simulated.

[0120] The "Movement Type" field contains information about the type of movement of the space robot being simulated.

[0121] The item "Movement Type" may include the following information regarding the type of movement of the space robot being simulated: • Does it operate inside the ship or outside the ship? • Does it float and move within the spacecraft, or does it move along the walls? (For example, if it moves along the walls of a spacecraft, it may avoid areas that should be avoided as part of its movement path, such as cables.) The "Purpose" item contains information about the intended use of the space robot being simulated.

[0122] The item "Purpose" may include the following as information about the purpose: • Sensing. By having the space robot itself perform sensing while moving, it is possible to expand the range and targets that can be sensed by the spacecraft, in addition to the sensors on the spacecraft itself. • Possible tasks. For example, if a space robot is equipped with an arm, it can perform tasks such as grasping and rotating objects, allowing it to carry, push, rotate, and repair objects within a spacecraft. The "Specifications" item contains information about the specifications of the space robot being simulated, such as its size and functions.

[0123] The item "Specifications" may include the following as information regarding the specifications of the space robot: Indicators indicating the size and weight of a space robot, such as mass (total weight at launch, total weight in space, etc.) and dimensions (overall size, shape). • Specifications of the propulsion system for attitude control (movement speed, method of movement, etc.) • Functions (targets of sensing by the space robot, sensing accuracy, tasks possible with the arms, etc., of the space robot) • Power consumption (power consumed by the movement of the space robot, rechargeable capacity, etc.) The item "3D Model Data" contains information about the data file of the 3D model data of the space robot that will be the subject of the simulation.

[0124] The item "3D Model Creator" is information that identifies the creator of the 3D model data for the space robot that is the subject of the simulation.

[0125] The item "3D Model Creator" may be associated with the item "User ID" in user database 211.

[0126] The item "3D Model Registration Date" indicates the timing when information such as the 3D model data of the space robot to be simulated was registered.

[0127] The item "3D Model Usage Conditions" contains information on the conditions for using the 3D model data of the space robot that will be the subject of the simulation.

[0128] More specifically, the item "3D Model Usage Conditions" includes information such as usage conditions set by the creator of the space robot and conditions set by the service provider that provides the service via Server 20. For example, conditions for using the 3D model data of a space robot may include paying the fee set by the person who set the conditions, participating in a space project, having a certain level of experience in space development, and having personnel with experience in space development on staff.

[0129] Figure 7 shows the data structure of the constraint database 214. The constraint database 214 includes the fields "Constraint ID", "Major Category", "Medium Category", "Minor Category", and "Constraint".

[0130] The item "Constraint ID" is information that identifies each constraint in space development.

[0131] The "Major Category" item indicates the broad category of the constraints.

[0132] Specifically, the "Major Category" item includes information on constraints for the following categories. There may be stricter performance constraints than for ground-based spacecraft (e.g., communication speed constraints). Also, the devices that can be adopted and the performance requirements may differ depending on the mission to be achieved with the spacecraft. Furthermore, conditions may differ depending on the space environment, such as inside the ISS, outside the ISS, or on the lunar surface. For example, the requirements for the environment inside the ISS include requirements for volume, power, communication, heat dissipation, gas disposal, and gas supply. Similarly, the requirements for the environment outside the ISS include requirements for power, communication, heat dissipation, and vibration environment. • Gravity constraint: The gravity is extremely small compared to that on Earth. For example, gravity inside and outside the ISS is on the order of 10 to the power of minus 6 compared to Earth, and on the lunar surface it is on the order of one-sixth, so the control of device movement may differ from that of ground-based vehicles. • Temperature constraint: Temperature changes can be significant compared to ground level. For example, while the temperature inside the ISS is assumed to be around 18 to 27 degrees Celsius, outside the ISS, it can be around -150 degrees Celsius in the shade and +120 degrees Celsius in sunlight. On the lunar surface, it can be around -170 degrees Celsius at night and +110 degrees Celsius during the day. The requirements for devices to withstand temperature changes that are not anticipated for ground-based devices may differ from those for ground-based devices. Furthermore, outside the ISS, heat from direct light, reflected light, infrared radiation, and cosmic microwave background radiation must be considered. The dew point inside the ISS is set to be approximately 4°C to 16°C. Communication constraints For example, the bands, communication capacity, and communication ports that can be used by spacecraft are defined and may differ from those of ground-based spacecraft. The communication environment may also be slower compared to ground-based spacecraft. • Power constraints For example, the total power consumption of a spacecraft may be determined according to the specific spacecraft. • Atmospheric constraints: Space approaches a vacuum.

[0133] For example, the atmospheric pressure inside the ISS is approximately 98-103 kPa, similar to that on Earth, while outside the ISS it is 10⁻⁵ Pa, and on the lunar surface it is zero Pa (extreme vacuum). The partial pressure of oxygen inside the ISS is sometimes set to approximately 20-23 kPa. The partial pressure of carbon dioxide inside the ISS may be kept below 707 Pa. The relative humidity inside the ISS is sometimes set to 25-70% RH. The circulating air velocity inside the ISS is sometimes set to approximately 0.07 to 0.2 m / s. • Constraints on radiation: There may be constraints such as plasma, ionizing radiation, and electromagnetic waves.

[0134] For example, regarding plasma, outside the ISS, the electron density is on the order of 10¹² electrons / cubic meter on the sunlit side. This can cause static charge, discharge, surface damage to materials, and malfunction of equipment. For example, ionizing radiation can cause malfunctions outside the ISS. For example, electromagnetic waves outside the ISS could potentially cause deterioration of parts and materials, and discoloration of paints and other materials. The "Subcategory" item indicates a category within a medium-level framework that is further subdivided within the broader category of constraints.

[0135] The "Subcategory" item provides information indicating more detailed categories that are further subdivided within the medium-level constraint framework.

[0136] Specifically, the "subcategory" item may be divided into categories such as "inside the ISS" and "outside the ISS," as shown in the diagram.

[0137] The item "Constraints" is information that indicates the constraints that are determined based on the factors that affect devices in space development.

[0138] Specifically, the item "Constraints" may hold the parameters that constitute constraints in each environment as described above.

[0139] Figure 8 shows the data structure of the command content database 215. The command content database 215 includes the items "Command Log ID", "Spacecraft ID", "Space Robot ID", "Sensing Result", "Robot Position and Attitude", "Command Content", "Control Parameters", and "Time Information".

[0140] The "Command Log ID" field is information that identifies each command used to control the space robot in the spacecraft being simulated.

[0141] The "Spacecraft ID" field is information that identifies each spacecraft being used in the simulation.

[0142] The item "Spacecraft ID" may be associated with the item "Spacecraft ID" in Spacecraft Database 212.

[0143] The "Space Robot ID" field is information that identifies each space robot being used in the simulation.

[0144] The item "Space Robot ID" may be associated with the item "Space Robot ID" in robot database 213.

[0145] The item "Sensing Results" refers to the information obtained from sensing by a space robot in the simulation environment.

[0146] More specifically, the item "Sensing Results" refers to sensing results such as temperature, humidity, cosmic radiation, and camera image data.

[0147] The item "Robot's Position and Orientation" contains information resulting from the space robot's estimation of its own position and orientation within the simulation environment.

[0148] More specifically, the item "Robot's Position and Orientation" includes the results of estimating the robot's position and orientation within the spacecraft based on image data captured by the robot inside the spacecraft in the simulation environment, and information on the shape of the spacecraft's interior.

[0149] The item "Command Content" contains information about the commands used to perform control in response to the sensing results.

[0150] The item "Instruction Content" may include the following as types of instructions: • Tasks performed by robotic arms, etc., on space robots (e.g., grasping objects, rotating objects, operating control panels of various devices, repair work, etc.) • Thermal control: Thermal control of various devices (e.g., thermal control to cope with temperature changes in outer space), temperature and humidity control suitable for human stays inside spacecraft, etc. • Movement control, attitude control: Setting movement paths for space robots to move according to commands, attitude control for movement, and driving the propulsion system. Some of these commands are specified by humans (for example, a space robot moves an object inside the spacecraft on behalf of a human, grasps it, and returns). Other times, commands for space robots are generated according to the control of the spacecraft (for example, the spacecraft sets a sensing schedule and instructs the space robot to perform sensing at designated locations). The item "Control Parameters" contains information about the parameters that drive the space robot, enabling the execution of commands to the space robot in the simulation environment.

[0151] More specifically, the item "Control Parameters" includes parameters for drive systems such as thrusters that move the space robot along a set travel path.

[0152] The "Time Information" item contains information about the timing of when the space robot operates according to commands in the simulation environment.

[0153] Figure 9 shows the data structure of the physical simulation results database 216. The physical simulation results database 216 includes the items "Simulation Log ID", "Spacecraft ID", "Space Robot ID", "Object Position and Attitude", "Environmental Conditions", "Command Content", "Physical Simulator Simulation Results", and "Parameters for Rendering Engine".

[0154] The "Simulation Log ID" field is information that identifies each log of a simulation that involved physical calculations.

[0155] The "Spacecraft ID" field is information that identifies each spacecraft being used in the simulation.

[0156] The item "Spacecraft ID" may be associated with the item "Spacecraft ID" in Spacecraft Database 212.

[0157] The "Space Robot ID" field is information that identifies each space robot being used in the simulation.

[0158] The item "Space Robot ID" may be associated with the item "Space Robot ID" in robot database 213.

[0159] The item "Object Position and Orientation" contains information resulting from the space robot's estimation of its own position and orientation within the simulation environment.

[0160] The "Environmental Conditions" item contains information about the conditions that affect spacecraft and space robots in the simulation environment.

[0161] The item "Environmental conditions" may include the following as conditions that affect spacecraft and space robots: • Microgravity. For example, gravity can have an effect on movement inside a space robot. • Airflow. For example, the movement of space robots and people can affect the airflow inside a spacecraft, creating air currents that can impact various pieces of equipment inside the spacecraft. ·temperature ·Humidity • Illumination. For example, when a space robot takes a picture with a camera in a simulation environment to estimate its own position (rendering based on the settings of a virtual camera corresponding to the camera), the captured data is generated according to the illumination conditions (too bright, too dark, etc.). The item "Command Content" contains information about the content of the commands given to the space robot in the simulation environment.

[0162] The item "Instruction Content" may be associated with the item "Instruction Content" in the instruction content database 215.

[0163] The item "Physical Simulator Simulation Results" contains information on the simulation results when the physical simulation of airflow and other factors associated with the movement of a space robot was performed using the physical simulation service server 97.

[0164] The item "Parameters for the rendering engine" contains information about the generated parameters for the rendering engine, which are created by abstracting the simulation results from server 97 of the physical simulation service.

[0165] The item "Parameters for the rendering engine" may include the following as a result of generating parameters for the rendering engine through abstraction: • In the physical simulation service server 97, to simulate airflow, the three-dimensional space inside the spacecraft is divided and calculations are performed. When calculation results are generated for each divided three-dimensional space, the number of divisions in the three-dimensional space is reduced to obtain a value to be passed to the rendering engine's physical simulator. For example, if the divided three-dimensional space is considered as blocks, the physical simulation service server 97 integrates the simulation results of adjacent blocks. For example, it may select a representative value from the simulation results of adjacent blocks (e.g., the direction and strength of the airflow generated as the space robot moves) or perform processing such as averaging the calculation results of multiple blocks. These airflow simulation results may affect the movement of multiple space robots inside the ship. Furthermore, in the simulation results from the physical simulation service server 97, it is also possible to select the wind flow with the greatest influence and disregard the other wind flows.

[0166] <3 operations> Figure 10 shows the process flow in which a computational fluid dynamics simulation is performed using a physical simulator, and then parameters to be applied to the rendering engine are generated from the simulation results, and rendering is performed.

[0167] In step S1021, the rendering processing module 2046 of the server 20 refers to the spacecraft database 212 and the robot database 213 and outputs to the terminal 10 the rendering result of a virtual space, which includes objects representing the interior space of the spacecraft, drawn by the rendering engine for the spacecraft to be rendered.

[0168] In this way, the rendering processing module 2046 provides the user of terminal 10 with a virtual space that is rendered by the rendering engine and includes a first object representing the interior space of the spacecraft.

[0169] In step S1011, terminal 10 receives operations from the user, including commands for actions on objects corresponding to robots on board the ship. For example, it receives commands from the user to move the robot, to move to a specific location on the ship and perform a specific action using the robot arm, etc., and to perform sensing.

[0170] In step S1023, the rendering processing module 2046 of the server 20 receives an action command for the robot object and updates the command content database 215.

[0171] In step S1025, the rendering processing module 2046 of the server 20 generates drive control parameters for making the robot move, work, etc., based on the content of the operation command to the robot, parameters of environmental conditions including gravity in the space environment of the spacecraft (item "environmental conditions" in the physical simulation result database 216), and information on the robot's specifications (weight, size, etc.).

[0172] Thus, the rendering module 2046 performs a physical simulation by applying environmental conditions, including microgravity in outer space. Here, the environmental conditions include the illumination conditions of the first object. The rendering module 2046 may also apply the illumination conditions to generate image data based on the position of the second object (rendering is performed based on the virtual camera settings according to the position and attitude of the second object). The rendering module 2046 may also generate position estimation parameters that estimate the position of the second object inside the first object based on the image data generated by applying the illumination conditions. For example, depending on whether the illumination is too bright or too dark, the camera settings may produce images that are too bright or too dark. In this case, when processing to estimate the position and attitude of the second object from the camera's captured image, it may not be possible to estimate the position and attitude inside the ship, and an error may be output.

[0173] Thus, the illumination may vary depending on the spacecraft's position in space (it may be in the shadow of sunlight), and the amount of light emitted from the spacecraft's lighting may also be adjusted (the amount of light inside the spacecraft may be reduced due to power supply and demand. Also, it is possible that there are no people on board the spacecraft and no lighting is used, with space robots handling sensing and maintenance). Based on these illumination conditions, it is possible to simulate whether a space robot can estimate its own position and attitude based on images captured by cameras.

[0174] In step S1027, the physical simulator control module 2047 of server 20 sends an instruction to the physical simulation service server 97 to execute a computational fluid dynamics physical simulation using the generated control parameters. The physical simulation provided by the physical simulation service server 97 is more accurate than the physical calculations provided by the rendering engine.

[0175] In this way, the physical simulator control module 2047 provides the physical simulator (physical simulation service server 97) that performs the physical simulation with information that controls the behavior of a second object, which is different from the first object and is movably positioned inside the first object, thereby causing the physical simulator to simulate physical phenomena affecting at least one of the first object and the second object.

[0176] The physical simulator control module 2047 may also include providing the physical simulator (physical simulation service server 97) with information that controls the operation of the second object, which includes at least one of the following: information on the specifications of the second object, including at least one of the shape and weight of the second object (robot database 213), or information on driving the second object (item "control parameters" in the command content database 215).

[0177] Here, the physical simulator (physical simulation service server 97) performs simulations using computational fluid dynamics. The physical simulator control module 2047 causes the physical simulator, which performs simulations using computational fluid dynamics, to perform simulations including the airflow inside the first object, thereby simulating the physical phenomena affecting at least one of the first object and the second object based on the airflow simulation results.

[0178] For example, the physics simulator control module 2047 uses a physics simulator that performs computational fluid dynamics simulations to divide the three-dimensional space inside the first object into multiple blocks and simulate physical phenomena in each block.

[0179] In step S1029, the rendering processing module 2046 of server 20 receives the results of a computational fluid dynamics physical simulation from the physical simulator (physical simulation service server 97) and generates parameters to be provided to the rendering engine. Based on the generated parameters, the rendering processing module 2046 updates the physical simulation results database 216 (item "Parameters for the rendering engine").

[0180] Here, the rendering processing module 2046 generates parameters to be given to the rendering engine so that the physical simulator can be executed by reducing the resolution in 3D space (also called reducing the granularity or abstracting) of the simulation results from the physical simulator (physical simulation service server 97).

[0181] For example, the rendering processing module 2046 may generate parameters to be given to the rendering engine by extracting high-priority airflow values ​​based on the magnitude of the airflow, based on the results of a simulation including airflow performed by a physical simulator (physical simulation service server 97).

[0182] Alternatively, the rendering processing module 2046 may use a physical simulator (physical simulation service server 97) that performs computational fluid dynamics simulations to divide the interior of the first object into multiple blocks, simulate physical phenomena in each block, and then, based on the calculation results of the simulation of physical phenomena in each block, integrate the calculation results of multiple blocks to obtain a simulation result with a reduced number of blocks, thereby generating parameters to be given to the rendering engine.

[0183] In step S1031, the rendering processing module 2046 of the server 20 refers to the physical simulation results database 216 and, based on the simulation results from the physical simulator (physical simulation service server 97), operates the object using the physical simulation function of the rendering engine, and sequentially outputs the rendering results to the terminal 10.

[0184] Thus, the rendering module 2046 uses the parameters generated in step S1029 and the physical simulation functions provided by the rendering engine to perform a physical simulation of the movement of the second object in the virtual space. The rendering module 2046 performs the physical simulation by applying environmental conditions, including microgravity in outer space.

[0185] Here, the second object could be an object corresponding to a floating robot that floats and moves inside the spacecraft. The rendering module 2046 makes the robot float and move within the first object according to the position estimation parameters generated for the robot.

[0186] Furthermore, the second object may be an object corresponding to a wall-climbing robot that moves along the wall surface of the spacecraft. The rendering processing module 2046 determines a path for moving the second object along the wall surface within the first object, according to the position estimation parameters generated for the robot. For example, there may be areas on the wall surface that the robot is not allowed to enter (e.g., areas where code is placed inside the spacecraft). In this case, the rendering processing module 2046 determines a path for moving the second object along the wall surface, avoiding the areas that should not be entered. The rendering processing module 2046 then moves the second object along the wall surface according to the determined path.

[0187] In step S1013, terminal 10 displays the rendering result of the object's movement on display 132.

[0188] <4 Screen Examples> Figure 11 shows an example of an operation screen that displays the results of a physical simulation performed according to environmental conditions in outer space.

[0189] The simulation operation screen 1100 is an operation screen that provides the results of rendering a 3D model of a spacecraft using a rendering engine, as part of the spacecraft simulation environment.

[0190] In the illustrated example, the simulation operation screen 1100 moves objects corresponding to robots using physics calculations based on conditions such as gravity in outer space. In this case, high-resolution simulations using computational fluid dynamics, including airflow, are performed by the physics simulation service server 97. However, instead of directly applying the high-resolution simulation results from the physics simulation service server 97 to the rendering engine's physics calculations, the calculation results from the physics simulation service server 97 are abstracted (for example, by reducing the resolution of the data in 3D space), making it easier for the rendering engine to use physics calculations to control the movement of objects (for example, the effect of airflow inside the spacecraft on the spacecraft itself, and the effect on other robots moving inside the spacecraft).

[0191] The account display area 1102 is the area that displays the accounts of users who use the simulation.

[0192] The simulation condition display area 1104 is an area that displays the conditions that affect each object in the simulation environment.

[0193] In the illustrated example, the simulation condition display area 1104 displays the name of the spacecraft in the simulation environment, the spacecraft's orbit, and the environmental conditions (gravity, etc.) affecting the spacecraft. In addition, the simulation settings may also display the spacecraft's specifications, information on space robots operating on the spacecraft, and devices where malfunctions are occurring. For example, depending on the setting of the spacecraft's orbit, the effects of gravity and sunlight (illuminance, temperature, etc., including whether it is in the shade) may be set as simulation conditions as the spacecraft moves along its orbit. Thus, the simulation condition display area 1104 may accept operations from the user to specify conditions that affect the movement of objects in the simulation environment, for example, by accepting the specification of the orbit. The server 20 may set conditions such as the effects of gravity and the period of time when it is in the shade according to the orbit specified by the user and provide the simulation environment as a virtual space.

[0194] The spacecraft designation unit 1106 is an operating component that accepts the designation of the spacecraft object to be used in the simulation.

[0195] In the illustrated example, the spacecraft designation unit 1106 displays the following monitoring results. The spacecraft 50 may also display the monitoring results for both inside and outside the spacecraft in the monitoring result display area 1306. • Sensing results related to thermal control (temperature of each device, temperature and humidity inside the ship, etc. Whether the temperature is suitable for the device to operate normally, whether the device is within temperature constraints, and whether the temperature and humidity are suitable for people to stay in). • Sensing results related to attitude control / orbit control (e.g., whether the car is on the planned orbit) • Sensing results related to power supply and control (power generation amount, power consumption amount, etc.) • Sensing results related to communication (e.g., actual communication volume) • Sensing results related to life support (sensing results of the human body's health status, inventory of water, food, etc. necessary for life support) The in-ship 3D model display area 1108 is an area that displays information about other space objects distinct from the spacecraft 50.

[0196] In the illustrated example, the in-ship 3D model display area 1108 accepts user input, for example, to specify a 3D model corresponding to a spacecraft. The server 20 refers to the spacecraft database 212, retrieves the 3D model data for the spacecraft specified by the user, and provides a virtual space inside the spacecraft.

[0197] Robot object 1110 is a user-operable object placed in a virtual space corresponding to the interior of a spaceship.

[0198] In the illustrated example, the robot object 1110 is the object corresponding to the robot. In the illustrated example, for illustrative purposes, an arrow indicating the direction of movement is shown near the robot object 1110, assuming it is moving inside a spacecraft. The server 20 may also display the direction of movement of such objects. This makes it even easier to understand how the robot object 1110 will move according to the user's commands.

[0199] The robot information display area 1112 is an area that displays information about the robot corresponding to the robot object 1110.

[0200] In the illustrated example, the robot information display area 1112 displays the following: • Robot specifications (dimensions, weight, functions of arms, etc.) • Based on the image data captured by the camera mounted on the robot (image data captured by a virtual camera in a virtual space according to the illumination settings), the position and attitude inside the spacecraft were estimated (estimated current position). • Is the image data itself properly captured? (It's possible that the image is too bright or too dark, preventing the interior of the spacecraft from being captured clearly enough to pinpoint its location.) • Content of commands to the robot (those specified by the user (e.g., move, move around inside the spacecraft and perform predetermined actions such as operating equipment), commands pre-set without user specification (e.g., the robot itself has sensing capabilities and will move around inside the spacecraft to perform sensing), commands set according to the status of the spacecraft (e.g., assuming a malfunction has occurred, move to check the location of the malfunction)). • Estimated work time required to perform the tasks in accordance with the instructions (estimated travel time and estimated work time based on the nature of the work) • Movement route (If the robot floats inside the ship, the route it will take by floating. If the robot crawls along walls, the route will avoid areas on the walls that it cannot enter.) • Status of the work (not started, in progress, completed, etc.) The robot designation unit 1114 is an operating component that accepts the designation of an object corresponding to a robot in the virtual space inside the spacecraft, which serves as the simulation environment.

[0201] In the illustrated example, the robot designation unit 1114, in response to user input, refers to the robot database 213 to receive a designation of a robot object to be placed inside the spaceship, retrieves the 3D model data of the designated robot, and places it in the virtual space inside the spaceship. Here, the placement of the robot object may be conditional on fulfilling the usage conditions of the robot object (item "3D model usage conditions" in the robot database 213) (for example, by performing an operation that satisfies the conditions, such as making a payment or agreeing to the usage conditions).

[0202] The command designation unit 1116 is an operating member that receives the designation of commands to be placed in the simulation environment for the robot.

[0203] In the illustrated example, the command designation unit 1116 may accept the following commands from the user. • Commands related to the movement and control of robot components, such as movement and robot arm control. - A command that indicates the purpose or objective, such as "bring an object from a designated location inside the spacecraft" (transport) or "perform a designated task at a designated location inside the spacecraft" (task). Server 20 converts this purpose / objective command into specific control content for the robot (for example, determining a movement route to the designated location, providing drive parameters for movement along the movement route, etc.). For example, the server 95 of the large-scale language model service may be prompted to generate the content and procedures for driving the robot according to the robot command specified by the user, thereby generating the content and procedures for driving the robot and managing the tasks corresponding to these.

[0204] The physical simulator confirmation operation unit 1118 is an operation component that receives operations to confirm information from the server 97 of a physical simulation service that performs simulations of airflow, etc., using computational fluid dynamics.

[0205] In the illustrated example, the physical simulator verification operation unit 1118 may display the accuracy of the simulation, other functions, operating status, and usage conditions of the physical simulation service server 97 in response to user operations.

[0206] The robot provider verification operation unit 1120 is an operation member that receives an operation to verify the information of the provider of the robot object.

[0207] In the illustrated example, the robot provider verification operation unit 1120, in response to user operations, refers to the robot database 213 (item "3D model creator") and displays information about the provider of a robot object already placed in the virtual space. For example, in response to operations on the robot provider verification operation unit 1120, it may accept requests from the provider for tasks related to the robot corresponding to the robot object (such as the design and manufacture of all or part of the components, parts, etc.).

[0208] The object manipulation unit 1122 is an operating member that receives operations on the robot object 1110 placed in the simulation environment.

[0209] In the illustrated example, the object manipulation unit 1122 receives commands from the user for the robot object 1110. For example, it displays an operating member for moving the robot object 1110, and in response to the operation of the operating member, it can move the robot object 1110 in the virtual space using physical calculations according to conditions such as gravity, while referring to the physical simulation result database 216 and utilizing the simulation results from the physical simulation service server 97.

[0210] <Variation> The matters described in the above embodiments may be combined in various ways.

[0211] <Other> A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0212] Furthermore, by distributing all or part of each hardware configuration across multiple computers and connecting them to each other via a network, a computer can be virtually realized. Thus, the concept of a computer includes not only computers housed in a single enclosure or case, but also virtualized computer systems.

[0213] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0214] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0215] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0216] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0217] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0218] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0219] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0220] (Note) The details described in each of the above embodiments are noted below.

[0221] (Note 1) A program for operating a computer having one or more computer processors, the program causing one or more computer processors to perform the following steps: providing a virtual space to one or more computer processors, which is a virtual space rendered by a rendering engine and includes a first object representing the interior space of a spacecraft; providing a physical simulator that performs physical simulations with information that controls the behavior of a second object, which is different from the first object and is movably positioned inside the first object, thereby causing the physical simulator to perform a simulation of physical phenomena affecting at least one of the first object and the second object; generating parameters to be given to the rendering engine by performing an abstraction process on the simulation results from the physical simulator; and performing a physical simulation in the virtual space in conjunction with the behavior of the second object using the generated parameters and the physical simulation functions provided by the rendering engine.

[0222] (Note 2) The program as described in Appendix 1, wherein, in the step of causing a physical simulator to simulate a physical phenomenon, the program provides the physical simulator with at least one of the following information as information to control the behavior of the second object: information regarding the specifications of the second object, which includes at least one of the shape and weight of the second object, or information to drive the second object.

[0223] (Note 3) The program according to any one of Appendix 1 to 2, wherein in the step of having the physical simulator perform a simulation of physical phenomena, the physical simulator, which performs a simulation using computational fluid dynamics, is made to perform a simulation including the airflow inside the first object, thereby simulating physical phenomena affecting at least one of the first object and the second object based on the simulation results of the airflow.

[0224] (Note 4) The program described in Appendix 3 generates parameters to be given to the rendering engine by extracting high-priority airflow values ​​based on the magnitude of the airflow, based on the results of a simulation including airflow from a physical simulator, in the step of generating parameters.

[0225] (Note 5) The program according to any one of the appendices 3 to 4, wherein in the step of having the physical simulator perform a simulation of a physical phenomenon, the physical simulator, which performs a simulation using computational fluid dynamics, has the interior of the first object divided into multiple blocks and has the physical phenomenon simulated in each of the blocks, and in the generation step, based on the calculation results of the simulation of the physical phenomenon in each of the blocks, the calculation results of the multiple blocks are integrated to obtain the simulation calculation result with a reduced number of blocks, thereby generating parameters to be given to the rendering engine.

[0226] (Note 6) A program described in any of the appendices 1 to 5, which performs a physical simulation in a virtual space by applying environmental conditions including microgravity in outer space.

[0227] (Note 7) The environmental conditions include the illuminance conditions of the first object, and the program, as described in Appendix 6, causes the processor to further apply the illuminance conditions to generate imaging data based on the position of the second object.

[0228] (Note 8) The program described in Appendix 7 generates position estimation parameters that estimate the position of a second object inside a first object, based on the image data generated by applying illumination conditions.

[0229] (Note 9) The second object is an object equivalent to a floating robot that floats and moves inside the spacecraft, and the program described in Appendix 8 causes it to float and move within the first object according to the generated position estimation parameters.

[0230] (Note 10) The second object is an object corresponding to a wall-climbing robot that moves along the wall surface of a spacecraft, and the program described in any of appendices 8 to 9 determines a path for moving the second object along the wall surface within the first object according to the generated position estimation parameters, and moves the second object along the wall surface according to the determined path.

[0231] (Note 11) A method of operating a computer comprising one or more computer processors, the method comprising: the one or more computer processors providing a virtual space rendered by a rendering engine, the virtual space including a first object representing an interior space of a spacecraft; providing information for controlling the operation of a second object, different from the first object, disposed movably inside the first object, to a physical simulator that performs a physical simulation, causing the physical simulator to simulate a physical phenomenon exerted on at least one of the first object and the second object; generating parameters to be provided to the rendering engine by performing a process of abstracting the simulation result by the physical simulator; and performing a physical simulation associated with the operation of the second object in the virtual space using the generated parameters and a physical simulation function provided in the rendering engine.

[0232] (Appendix 12) An information processing apparatus, wherein a control unit of the information processing apparatus provides a virtual space rendered by a rendering engine, the virtual space including a first object representing an interior space of a spacecraft; provides information for controlling the operation of a second object, different from the first object, disposed movably inside the first object, to a physical simulator that performs a physical simulation, causing the physical simulator to simulate a physical phenomenon exerted on at least one of the first object and the second object; generates parameters to be provided to the rendering engine by performing a process of abstracting the simulation result by the physical simulator; and performs a physical simulation associated with the operation of the second object in the virtual space using the generated parameters and a physical simulation function provided in the rendering engine.

Claims

1. A program for operating a computer having one or more computer processors, The program is configured on one or more computer processors. The steps of providing a virtual space which is rendered by a rendering engine and includes a first object representing the interior space of a spacecraft, The steps include providing a physical simulator that performs a physical simulation with information that controls the behavior of a second object, which is different from the first object and is movably positioned inside the first object, thereby causing the physical simulator to perform a simulation of physical phenomena affecting at least one of the first object and the second object, The steps include generating parameters to be provided to the rendering engine by performing an abstraction process on the simulation results from the aforementioned physical simulator, A program that performs the steps of: performing a physical simulation in the virtual space, corresponding to the movement of the second object, using the generated parameters and the physical simulation functions provided by the rendering engine.

2. In the step of having the physical simulator perform a simulation of the physical phenomenon, the information used to control the behavior of the second object is as follows: Information relating to the specifications of the second object, the information relating to the specifications of the second object, which includes at least one of the shape and weight of the second object. or Information that drives the second object, The program according to claim 1, comprising providing at least one of the following pieces of information to the physical simulator.

3. In the step of having the aforementioned physical simulator perform a simulation of a physical phenomenon, The program according to claim 1, wherein the physical simulator that performs computational fluid dynamics simulations is used to perform a simulation including the airflow inside the first object, thereby simulating physical phenomena affecting at least one of the first object and the second object based on the simulation results of the airflow.

4. The program according to claim 3, wherein in the step of generating the parameters, the program generates the parameters to be given to the rendering engine by extracting airflows with high priority based on the magnitude of the airflows, based on the results of a simulation including the airflows performed by the physical simulator.

5. In the step of having the aforementioned physical simulator perform a simulation of a physical phenomenon, The physical simulator, which performs simulations using computational fluid dynamics, divides the interior of the first object into multiple blocks and simulates the physical phenomenon in each block. The program according to claim 3, wherein in the generation step, parameters to be given to the rendering engine are generated by integrating the calculation results of a plurality of blocks based on the calculation results of the simulation of the physical phenomenon of each block, thereby reducing the number of blocks and obtaining the simulation calculation results.

6. The program according to claim 1, wherein in the step of performing the physical simulation in the virtual space, the program performs the physical simulation by applying environmental conditions including microgravity in outer space.

7. The aforementioned environmental conditions include the illuminance conditions for the first object. The program further provides the processor with: The program according to claim 6, which generates imaging data based on the position of the second object by applying the aforementioned illumination conditions.

8. The program according to claim 7, which generates position estimation parameters that estimate the position of the second object inside the first object based on the shooting data generated by applying the illumination conditions.

9. The second object is an object that corresponds to a floating robot that floats and moves inside the spacecraft, The program according to claim 8, which causes the first object to float and move according to the generated position estimation parameters.

10. The second object is an object that corresponds to a wall-climbing robot that moves along the wall surface of the spacecraft, Based on the generated position estimation parameters, a path is determined for moving the second object along the wall surface within the first object, The program according to claim 8, which moves the second object along the wall surface according to a determined path.

11. A method for operating a computer having one or more computer processors, The above method involves one or more computer processors, The steps of providing a virtual space which is rendered by a rendering engine and includes a first object representing the interior space of a spacecraft, The steps include providing a physical simulator that performs a physical simulation with information that controls the behavior of a second object, which is different from the first object and is movably positioned inside the first object, thereby causing the physical simulator to perform a simulation of physical phenomena affecting at least one of the first object and the second object, The steps include generating parameters to be provided to the rendering engine by performing an abstraction process on the simulation results from the aforementioned physical simulator, A method for performing the steps of: performing a physical simulation in the virtual space, corresponding to the movement of the second object, using the generated parameters and the physical simulation functions provided in the rendering engine.

12. An information processing device, The control unit of the information processing device, The steps of providing a virtual space which is rendered by a rendering engine and includes a first object representing the interior space of a spacecraft, The steps include providing a physical simulator that performs a physical simulation with information that controls the behavior of a second object, which is different from the first object and is movably positioned inside the first object, thereby causing the physical simulator to perform a simulation of physical phenomena affecting at least one of the first object and the second object, The steps include generating parameters to be provided to the rendering engine by performing an abstraction process on the simulation results from the aforementioned physical simulator, An information processing device that performs the steps of: performing a physical simulation in the virtual space, corresponding to the movement of the second object, using the generated parameters and the physical simulation functions provided in the rendering engine.

Images