Information processing system, information processing method, and program

Abstract

This invention provides an information processing system, an information processing method, and a program that can prioritize the execution of high-impact processes in the digital space. [Solution] An information processing system comprising: a real-space information acquisition unit that acquires real-space information indicating that the state of a real-space represented by a model in which a first object is placed in the real world has changed; a priority processing unit that determines the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information; a processing order control unit that sorts the execution order of processes performed in a digital space that reproduces the real-space by placing a second object corresponding to the first object, in order of priority based on the priority determination result; and a digital space management unit that executes the processes in the sorted execution order.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, technologies related to the digital space have been developed, and various technologies for linking the real space and the digital space have been proposed.

[0003] For example, Patent Document 1 below discloses a technique for controlling the display of an object in a virtual space and the change of the state of the object by using data acquired from a sensor device, a camera, etc. provided in the real space.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technique described in Patent Document 1 above, the priority regarding the display of an object in the virtual space is considered, but the priority including, for example, processing related to environmental changes is not considered up to that level. For this reason, for example, in digital twins and spatial computing, there is a risk that processing with a lower priority such as an effect on an individual object may be prioritized over processing with a higher priority that gives an important change to the entire virtual space, and there is a problem that the experience design deviates from the actual situation.

[0006] In view of the above problems, an object of the present invention is to provide an information processing system, an information processing method, and a program capable of preferentially executing processing with a high degree of influence in the digital space.

Means for Solving the Problems

[0007] To solve the above-mentioned problems, an information processing system according to one aspect of the present invention is an information processing system comprising: a real-space information acquisition unit that acquires real-space information indicating that the state of a real space represented by a model in which a first object is placed in the real world has changed; a priority processing unit that determines the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information; a processing order control unit that sorts the execution order of processes performed based on the real-space information in a digital space that reproduces the real space by placing a second object corresponding to the first object, in order of priority based on the priority determination result; and a digital space management unit that executes the processes in the sorted execution order.

[0008] An information processing method according to one aspect of the present invention is an information processing method performed by a computer, which includes: a real-space information acquisition process for acquiring real-space information indicating that the state of a real space represented by a model in which a first object is placed in the real world has changed; a priority processing process for determining the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information; a processing order control process for sorting the execution order of processes performed based on the real-space information in a digital space that reproduces the real space by placing a second object corresponding to the first object, in order of priority based on the priority determination result; and a digital space management process for executing the processes in the sorted execution order.

[0009] A program according to one aspect of the present invention is a program for causing a computer to function as: a real-space information acquisition means for acquiring real-space information indicating that the state of a real space represented by a model in which a first object is placed in the real world has changed; a priority processing means for determining the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information; a processing order control means for sorting the execution order of processes performed in a digital space that reproduces the real space by placing a second object corresponding to the first object, based on the priority determination result, in order of priority; and a digital space management means for executing the processes in the sorted execution order. [Effects of the Invention]

[0010] According to the present invention, processes with a high impact in the digital space can be executed with priority. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of the configuration of the information processing system according to this embodiment. [Figure 2] This figure shows an example of a model of the real space according to this embodiment. [Figure 3] This figure shows an example of an image within the virtual space as seen from the viewpoint of a virtual object placed in the virtual space according to this embodiment. [Figure 4] This is a branch office showing an example of the functional configuration of a real-world object according to this embodiment. [Figure 5] This is a block diagram showing an example of the functional configuration of the information processing device according to this embodiment. [Figure 6] This figure shows an example of priority information according to this embodiment. [Figure 7] This flowchart shows an example of the processing flow in the information processing device according to this embodiment. [Figure 8] This figure shows an embodiment according to this embodiment. [Figure 9]It is a diagram showing an example of priority information in a modification according to the present embodiment.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] <1. Configuration of Information Processing System> Referring to FIGS. 1 to 3, the configuration of the information processing system according to the present embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of the information processing system according to the present embodiment.

[0014] The information processing system 1 shown in FIG. 1 is a system for reproducing a real space represented by a model in the real world in a digital space and sorting and executing processes corresponding to information acquired from the real space based on priority.

[0015] The information processing system 1 reproduces, for example, a real space represented by a model MD (an example of a model) in a virtual space (an example of a digital space). Note that the information processing system 1 according to the present embodiment can be applied to, for example, a digital twin (particularly, a digital twin metaverse as a communication platform conscious of real-time interaction) that can represent the environment of the real world in a virtual space and perform simulations, or spatial computing conscious of robots including AI. Hereinafter, as an example, the present embodiment will be described using a digital twin to represent a hamster cage (an example of a real space) in the real world by a model MD and reproducing the hamster cage as a virtual space based on the model MD.

[0016] As shown in FIG. 1, the information processing system 1 includes a real object 10 (first object), a sensor device 20, an information processing device 30, and a display 40.

[0017] (1) Real Object 10 The real object 10 is an object existing in the real world. The real object 10 is also an object to be reproduced in the virtual space. The real object 10 may be, for example, the object itself existing in the real world, or a robot corresponding to the object existing in the real world. The objects existing in the real world include objects that cannot be controlled by a computer, such as humans and animals. The robot is a robot (agent) operable by computer control, a device (environmental node) for indicating an object that brings about an environmental change, and the like. Note that the real object 10 may be a stationary object that does not operate. Any object may be manually operated or moved by the user.

[0018] In the following, as an example, an example in which a small robot indicating a hamster and a small robot indicating a spinning wheel are used as agents for the real object 10, and a lamp indicating the sun or the moon is used as an environmental node will be described as an example of the present embodiment. The small robot performs operations such as moving, emitting light, expanding and contracting, vibrating, and rotating by autonomous driving. The real object 10 of the hamster includes, for example, a microcomputer, an actuator, and tires. Therefore, the real object 10 of the hamster can move (an example of an operation) by the actuator being controlled by the microcomputer and the tires being driven. In addition, the real object 10 of the spinning wheel includes, for example, a microcomputer, an actuator, and a spinning wheel provided rotatably. Therefore, the real object 10 of the spinning wheel can rotate the spinning wheel (an example of an operation) by the actuator being controlled by the microcomputer and the spinning wheel being driven. The real object 10 of the lamp indicates whether it is the sun or the moon depending on the color of lighting. For example, when the lamp is lit red, it indicates that it is the sun, and when the lamp is lit blue, it indicates that it is the moon. Thereby, when the lamp changes from red to blue, it indicates that the environment has changed from day to night, and when it changes from blue to red, it indicates that the environment has changed from night to day.

[0019] If the real-world object 10, which is equipped with communication capabilities, is a communicative object, it is connected to the information processing device 30 via a wired or wireless connection. In this embodiment, as an example, an example in which the real-world object 10 is connected to the information processing device 30 via a wireless connection will be described. The communication standard used by the real-world object 10 is, for example, BLE (Bluetooth Low Energy, Bluetooth is a registered trademark). In BLE communication, the real-world object 10 functions as a peripheral (slave), and the information processing device 30, which is the communication partner, functions as a central (master). As a result, if the real-world object 10 is an object that can be operated by computer control, its operation is controlled based on instructions received from the information processing device 30. In this embodiment, the hamster real object 10 and the hamster wheel real object 10 are equipped with communication functions, while the lamp real object 10 is not equipped with communication functions.

[0020] Real-world objects 10 are placed in the model MD. Real-world objects 10 are used to represent, for example, objects that operate in real space or objects that cause environmental changes in virtual space. Objects that operate in real space include, for example, people, animals, robots, and vehicles. Objects that cause environmental changes include, for example, the sun and the moon. One or more real-world objects 10-1 to 10-n (where n is a natural number) are placed in the model MD.

[0021] Now, with reference to Figure 2, the model MD of the real space according to this embodiment will be described. Figure 2 is a diagram showing an example of the model MD of the real space according to this embodiment.

[0022] The model MD shown in Figure 2 represents the interior of a hamster cage. The model MD contains four real-world objects 10. Real-world objects 10-1 and 10-2 are real-world objects representing hamsters. Real-world objects 10-1 and 10-2 are positioned to represent hamsters being kept in a cage. The hamster in real-world object 10-1 is assumed to be facing the direction indicated by arrow D1. Reality object 10-3 is a real-world object representing a hamster wheel. Reality object 10-3 is positioned with the assumption that it will be used by a hamster being kept in a cage. Reality object 10-4 is a reality object representing the sun or moon. Reality object 10-4 is positioned to indicate whether it is day or night inside the cage.

[0023] (2) Sensor device 20 The sensor device 20 is a device for acquiring environmental information indicating changes in the environment of the real space. The sensor device 20 may include, for example, a camera (image sensor), a temperature sensor, a humidity sensor, and an illuminance sensor. The camera is installed to capture images within the real space and is used to detect changes in the real space (for example, changes in the color of a lamp) through image recognition. The temperature sensor is used to detect temperature changes in the real space. The humidity sensor is used to detect humidity changes in the real space. The illuminance sensor is used to detect changes in illuminance in the real space.

[0024] The sensor device 20 is connected to the information processing device 30 via a wired or wireless connection, enabling communication between them. The sensor device 20 transmits environmental information indicating detected environmental changes to the information processing device 30.

[0025] (3) Information processing device 30 The information processing device 30 is composed of, for example, one or more PCs (Personal Computers) or server devices (e.g., cloud servers). In the following, this embodiment will be described using an example where the information processing device 30 is a PC. The information processing device 30 is connected to the real object 10, the sensor device 20, and the display 40 via wired or wireless connection, enabling communication between them.

[0026] The information processing device 30 is a device that reproduces the real space represented by a model in the real world in a virtual space (digital space), and sorts and executes processing corresponding to the information acquired from the real space based on priority. When reproducing the real space in the virtual space, the information processing device 30 creates a virtual object (second object) corresponding to each real object 10 placed in the model MD, and places it in the virtual space. The virtual object may be a direct representation of an object, or it may be represented by a character or avatar. The information processing device 30 transmits information for displaying the virtual space to the display 40, and displays the virtual space.

[0027] Assume that a real-world object 10 within the model MD is operated by user input or computer control. In this case, the information processing device 30 obtains (receives) object information from the operated real-world object 10 that indicates information about the real-world object 10. Based on the received object information, the information processing device 30 operates the corresponding virtual object in the virtual space in accordance with the operation of the real-world object 10. On the other hand, suppose a virtual object in the virtual space is operating. In this case, the information processing device 30 obtains object information from the operating virtual object in the virtual space, indicating information about that virtual object. Based on the obtained object information, the information processing device 30 operates the corresponding real object 10 in the model MD in accordance with the operation of the virtual object.

[0028] (4) Display 40 Display 40 is a display device for displaying a virtual space that reproduces the model MD. Display 40 is connected to the information processing device 30 by a wired or wireless connection. Display 40 displays the virtual space based on information for displaying the virtual space received from the information processing device 30. If multiple virtual objects exist in the virtual space, display 40 may display the images of the virtual space from the viewpoint of each virtual object together (in a listable format).

[0029] Now, referring to Figure 3, we will describe an image of the virtual space as seen from the viewpoint of a virtual object placed in the virtual space according to this embodiment. Figure 3 is a diagram showing an example of an image of the virtual space as seen from the viewpoint of a virtual object placed in the virtual space according to this embodiment.

[0030] The display 40 shown in Figure 3 shows a virtual space that recreates the model MD shown in Figure 2. As an example, the image displayed is an image of the virtual space as seen from the viewpoint of virtual object 11-1, which corresponds to real object 10-1, among the real objects 10-1 to 10-4 placed in the model MD shown in Figure 2 (the image as seen in the direction indicated by arrow D1 in Figure 2). Virtual object 11-2 is a virtual object that corresponds to real object 10-2, and virtual object 11-3 is a virtual object that corresponds to real object 10-3. In the example shown in Figure 3, since real object 10-4 in Figure 2 is not in the field of view of real object 10-1, virtual object 11, which corresponds to real object 10-4, is not displayed.

[0031] <2. Functional Configuration of Reality Objects> The configuration of the information processing system 1 according to this embodiment has been described above. Next, an example of the functional configuration of the real object 10 according to this embodiment will be described with reference to Figure 4. Figure 4 is a block diagram showing an example of the functional configuration of the real object 10 according to this embodiment. As shown in Figure 4, the real object 10 comprises a communication unit 110, a drive unit 120, a sensor unit 130, and a control unit 140.

[0032] (1) Communications Department 110 The communication unit 110 has the function of sending and receiving various types of information. The communication unit 110 is connected to the information processing device 30 by wired or wireless connection and sends and receives various types of information.

[0033] (2) Drive unit 120 The drive unit 120 has the function of operating the real object 10. When the communication unit 110 receives an operation instruction from the information processing device 30, the drive unit 120 is driven by the control unit 140 and operates the real object 10.

[0034] (3) Sensor unit 130 The sensor unit 130 has the function of acquiring information sensed by a sensor device provided on the real object 10 (hereinafter also referred to as "sensing information"). The real object 10 is equipped with a reading sensor on the bottom surface of its housing for acquiring position information. In this embodiment, a mat is laid on the floor of the model MD, and position information on the mat is printed on the surface of the mat. The sensor unit 130 acquires the position information of the real object 10 within the model MD by reading this position information with the reading sensor. Furthermore, the real-world object 10 may be equipped with sensor devices (for example, an accelerometer, a gyroscope, a motion sensor, etc.) for acquiring posture information. Furthermore, the real-world object 10 may be equipped with sensor devices (e.g., image sensors, distance sensors, etc.) for acquiring information about the environment surrounding its own device.

[0035] (4) Control unit 140 The control unit 140 has the function of controlling the overall operation of the real object 10. The control unit 140 is implemented, for example, by having a microcontroller or similar device, which is installed as hardware in the real object 10, execute a program. As shown in Figure 4, the control unit 140 includes a state detection unit 141.

[0036] (4-1) State detection unit 141 The state detection unit 141 has the function of detecting the state of the real object 10. For example, the state detection unit 141 detects whether or not there has been a change in the state of the real object 10 based on sensing information acquired by the sensor unit 130. If a change in state is detected, the state detection unit 141 transmits information indicating the content of the change in state (hereinafter also referred to as "state change information") to the information processing device 30. As an example, if it is detected that the real object 10 has moved, the state detection unit 141 transmits object information including state change information indicating that the real object 10 has moved to the information processing device 30.

[0037] <3. Functional Configuration of Information Processing Devices> The functional configuration of the real object 10 according to this embodiment has been described above. Next, an example of the functional configuration of the information processing device 30 according to this embodiment will be described with reference to Figures 5 to 6. Figure 5 is a block diagram showing an example of the functional configuration of the information processing device 30 according to this embodiment. As shown in Figure 5, the information processing device 30 includes a communication unit 310, an input unit 320, a storage unit 330, a control unit 340, and an output unit 350.

[0038] (1) Communications Section 310 The communication unit 310 has the function of sending and receiving various types of information. The communication unit 310 is connected to the real object 10, the sensor device 20, and the display 40 by wired or wireless connection, and sends and receives various types of information.

[0039] (2) Input section 320 The input unit 320 has the function of receiving input from the user. The function of the input unit 320 is realized, for example, by a mouse, keyboard, buttons, touch panel, microphone, etc., provided by the information processing device 30.

[0040] (3) Storage section 330 The storage unit 330 has the function of storing various types of information. The storage unit 330 is composed of storage media provided as hardware by the information processing device 30, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read / write Memory), ROM (Read Only Memory), or any combination of these storage media.

[0041] (4) Control unit 340 The control unit 340 has the function of controlling the overall operation of the information processing device 30. The control unit 340 can be implemented, for example, by causing the CPU (Central Processing Unit) or GPU (Graphics Processing Unit) provided as hardware in the information processing device 30 to execute a program. Alternatively, the control unit 340 may be implemented using software such as Unity® provided by Unity Technologies, Inc. or Unreal Engine® provided by Epic Games, Inc. As shown in Figure 5, the control unit 340 comprises a digital space management unit 341, a real space information acquisition unit 342, a digital space information acquisition unit 343, a priority processing unit 344, a processing order control unit 345, an operation control unit 346, and a display control unit 347.

[0042] (4-1) Digital Space Management Department 341 The Digital Space Management Unit 341 has the function of managing the virtual space (digital space). For example, the Digital Space Management Unit 341 generates a virtual space that reproduces the model MD, places virtual objects in the virtual space, detects changes in the state of the virtual objects, controls the movement of the virtual objects, and controls animation in the virtual space.

[0043] When the digital space management unit 341 performs processing on the virtual space, it executes the processing in the execution order sorted by the processing order control unit 345. For example, suppose the execution order is sorted to prioritize processing based on environment information. In this case, when environment information and object information are acquired, the digital space management unit 341 prioritizes executing processing based on environment information, and then executes processing based on object information.

[0044] Furthermore, processes that were not prioritized may be executed after acquiring real-world space information again at the time of execution. For example, suppose that when environment information and object information are acquired, the digital space management unit 341 prioritizes executing the process based on the environment information. In this case, the digital space management unit 341 uses the object information acquired again by the real-world space information acquisition unit 342 to execute the process based on the object information. This allows the digital space management unit 341 to execute processes that reflect the latest state of the virtual space.

[0045] (4-2) Real-world information acquisition unit 342 The real-space information acquisition unit 342 has the function of acquiring real-space information. Real-space information is information that indicates that the state of the real space has changed, and includes, for example, environment information and object information. Environment information is information that indicates that the environment of the real space has changed. Object information is information that indicates that the state of the real object 10 placed in the real space has changed. The real-space information acquired by the real-space information acquisition unit 342 includes at least one of environment information and object information.

[0046] When environmental information and object information are acquired, the processing performed by the digital space management unit 341 based on the environmental information is given priority. In this case, the real space information acquisition unit 342 acquires the object information again after the processing is completed.

[0047] (4-3) Digital spatial information acquisition unit 343 The digital space information acquisition unit 343 has the function of acquiring virtual space information (digital space information). Virtual space information is information that indicates that the state of the virtual object 11 has changed due to the execution of processing in the virtual space.

[0048] (4-4) Priority Processing Unit 344 The priority processing unit 344 has the function of performing processing related to priority. Based on priority information indicating the priority set for each piece of information acquired as real-space information, the priority processing unit 344 determines the priority of the real-space information acquired by the real-space information acquisition unit 342. The priority information is prepared as information in which the impact coefficient indicating priority is set as a fixed value in advance and is stored in the storage unit 330.

[0049] Now, with reference to Figure 6, the priority information according to this embodiment will be described. Figure 6 is a diagram showing an example of the priority information according to this embodiment.

[0050] As shown in Figure 6, priority information is associated with real-world spatial information and priority (impact coefficient), and may also be associated with the corresponding processing. For example, the "lamp color" acquired as environmental information within the real-world space data is associated with an impact coefficient of "0.9" and the processing action of "scene transition." Similarly, the "location information" acquired as object information within the real-world space data is associated with an impact coefficient of "0.5" and the processing action of "proximity event." In addition, some "location information" is associated with an impact coefficient of "0.7" and the processing action of "object movement."

[0051] (4-5) Processing sequence control unit 345 The processing order control unit 345 has the function of controlling the execution order of processes in the virtual space. The processing order control unit 345 sorts the execution order of processes that are executed in the virtual space based on real-world information in order of priority, based on the priority determination result by the priority processing unit 344.

[0052] (4-6) Operation control unit 346 The motion control unit 346 has the function of controlling the movement of objects. After processing by the digital space management unit 341, the motion control unit 346 controls the movement of the real object 10 corresponding to the virtual object 11 based on the virtual space information acquired by the digital space information acquisition unit 343. For example, if the virtual object 11 moves in the virtual space, the motion control unit 346 moves the corresponding real object 10 in the same way.

[0053] (4-7) Display control unit 347 The display control unit 347 has functions to control various displays. The display control unit 347 causes the display 40 to display a virtual space.

[0054] (5) Output section 350 The output unit 350 has the function of outputting various types of information. The output unit 350 is composed of output devices provided as hardware by the information processing device 30, such as display devices such as a display device or a touch screen (touch panel), and audio output devices such as a speaker.

[0055] <4. Processing Flow> The functional configuration of the information processing device 30 according to this embodiment has been described above. Next, the processing flow in the information processing device 30 according to this embodiment will be described with reference to Figure 7. Figure 7 is a flowchart showing an example of the processing flow in the information processing device 30 according to this embodiment.

[0056] As shown in Figure 7, first, the real-space information acquisition unit 342 of the information processing device 30 acquires real-space information (step S101). The real-space information acquired by the real-space information acquisition unit 342 may include only environment information, only object information, or both environment information and object information. In the following flowchart, we will describe an example where the real-space information acquisition unit 342 acquires both environmental information indicating the "color of the lamp" and object information indicating "location information" as real-space information.

[0057] Next, the priority processing unit 344 of the information processing device 30 determines the priority of the acquired real-space information (step S102). When using the priority information shown in Figure 6, the priority processing unit 344 determines that the priority of "lamp color" with an impact coefficient of "0.9" is the highest, followed by "location information" with an impact coefficient of "0.7", and then "location information" with an impact coefficient of "0.5", in descending order of priority.

[0058] Next, the processing order control unit 345 of the information processing device 30 sorts the execution order of the processes in descending order of priority (step S103). If the priority processing unit 344 determines the priority based on the priority information shown in Figure 6, the processing order control unit 345 sorts the execution order of the processes in the order of "scene transition", "object movement", and "approach event".

[0059] Next, the digital space management unit 341 of the information processing device 30 executes the processing in the sorted order (step S104). If the processing order control unit 345 has sorted based on the priority information shown in Figure 6, the digital space management unit 341 executes the processing in the order of "scene transition," "object movement," and "approach event."

[0060] The processing flow according to this embodiment has been described above. As described above, the information processing system 1 according to this embodiment includes: a real-space information acquisition unit 342 that acquires real-space information indicating that the state of the real space represented by a model MD (model) in which a real object 10 (first object) is placed in the real world has changed; a priority processing unit 344 that determines the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information; a processing order control unit 345 that sorts the execution order of processes executed based on the real-space information in a virtual space (digital space) that reproduces the real space by placing a virtual object 11 (second object) corresponding to the real object 10, in order of priority based on the priority determination result; and a digital space management unit 341 that executes the processes in the sorted execution order.

[0061] With this configuration, the information processing system 1 can execute processing on the virtual space, taking into consideration the priority of not only the processing related to the display of objects in the virtual space, but also the processing related to environmental changes. Therefore, the information processing system 1 according to this embodiment enables the priority execution of processes with a high impact in the digital space.

[0062] <5. Examples> This embodiment has been described above. Next, an example of the above-described embodiment will be explained with reference to Figure 8. Figure 8 is a diagram showing an example according to this embodiment.

[0063] As shown in Figure 8, first, assume that the color of the lamp's real object 10-4 in real space changes from red to blue (step S11). The sensor device 20 detects changes in the color of real-world objects 10-4, for example, by taking pictures with a camera (step S12). The sensor device 20 transmits environmental information indicating the detected change in the state of the real object 10-4 to the information processing device 30 (step S13). At this time, it is assumed that object information indicating the change in the state of the real object 10-1 is also transmitted from the hamster real object 10-1 to the information processing device 30.

[0064] The information processing device 30 sorts the execution order of processes in the virtual space based on the environmental information received from the sensor device 20 and the object information received from the real object 10-1 (step S14). In the example shown in Figure 8, the processes are sorted so that the processes associated with the state change of the lamp real object 10-4 are given priority over the processes associated with the state change of the hamster real object 10-1. The information processing device 30 executes processing on the virtual space according to the execution order after sorting (step S15).

[0065] As a result of processing by the information processing device 30, the virtual space is switched from day to night based on environmental information (step S16). Also, as the virtual space switches from day to night, the virtual object 11-1 of the nocturnal hamster is switched from a sleeping state to an awake state (step S17). The information processing device 30 acquires virtual space information indicating that the state of the virtual object 11-1 has changed in conjunction with the execution of processing on the virtual space (step S18).

[0066] The information processing device 30 controls the operation of the real object 10-1 corresponding to the virtual object 11-1 based on the acquired virtual space information (step S19). Through operation control by the information processing device 30, the hamster's real-world object 10-1 is switched from a sleeping state to an awake state in the real world based on the virtual space information (step S20). Thus, the day-night scene transition, which has a higher priority than processes such as aligning objects in the virtual space and playing animations, is performed first, followed by other processes such as displaying objects and playing animations corresponding to the scene. This is because it takes into account that the behavior of objects and the content of animations may differ depending on the scene.

[0067] The embodiments of this model have been described above. As described above, the information processing system 1 according to this embodiment sets a large impact coefficient for the time of day and night, which are common to the entire virtual space. For example, it can prioritize integrated changes over other environmental changes, such as transitioning agents in the virtual space to sleep mode and the virtual space itself to a scene that simulates a nighttime environment. Furthermore, even if processes would normally occur simultaneously, the information processing system 1 can prioritize those processes that should not occur simultaneously, allowing them to be processed sequentially rather than simultaneously. Furthermore, the execution order of processes can be easily changed simply by altering the priority (impact coefficient). This eliminates the need to extensively rewrite the program from time to time, making maintenance easier and reducing costs.

[0068] Another example involves applying the information processing system 1 to a digital twin that simulates a farm. In this case, weights are assigned to each variable, such as setting a high impact coefficient for temperature, humidity, and illuminance among the information acquired from the sensor device 20 in the real space, and setting a low impact coefficient for information indicating who passed in front of the sensor and for how long. This allows information important for livestock management to be assigned a high impact coefficient and to be processed with priority over other acquired information.

[0069] <6. Variation> Next, modifications of the above-described embodiment will be explained. Each modification described below may be applied to the embodiment individually or in combination. Furthermore, each modification may be applied in place of the configuration described in the embodiment, or it may be applied in addition to the configuration described in the embodiment.

[0070] In the embodiment described above, the real space is a hamster cage, the model representing the real space is a model MD, and the digital space is a virtual space. However, the embodiment is not limited to this example. For example, the real space may be a building, a park, a city, etc. The model representing the real space may be a map, etc. The digital space may be a mixed reality space in which the real space and digital information are superimposed, or a space that records the real world, such as Street View provided by Google.

[0071] Furthermore, in the embodiments described above, an example was described in which the hamster real object 10 and the hamster wheel real object 10 (i.e., agents) have communication functions, while the lamp real object 10 (i.e., environment node) does not have communication functions. However, the invention is not limited to such an example. For example, the real object 10 may include agents that do not have communication functions, or environment nodes that do have communication functions.

[0072] Furthermore, although the above-described embodiment mentions an example in which an external display 40 is used as the display device, the invention is not limited to this example. For example, the display of the information processing device 30 may be used instead of the external display 40.

[0073] Furthermore, although the above-described embodiment describes an example in which the information processing device 30 is connected to a communicable real object 10 via a wired or wireless connection and receives object information from the real object 10, the invention is not limited to such an example. For example, the information processing device 30 may acquire information sensed by various sensor devices about the real object 10 as object information from various sensor devices.

[0074] Furthermore, in the above-described embodiment, an example was explained in which the sensor unit 130 acquires the position information of the real object 10 within the model MD by reading position information printed on the surface of the mat with a reading sensor, but the invention is not limited to this example. For example, the sensor unit 130 may acquire the position information of the real object 10 by position estimation using techniques such as LiDAR (Light Detection And Ranging) or SLAM (Simultaneous Localization And Mapping).

[0075] Furthermore, although the above-described embodiment explained an example in which the impact coefficient indicating priority is a predetermined fixed value, the invention is not limited to such an example. For example, the impact coefficient may be a variable value defined by a function of at least one evaluation value. In this case, the impact coefficient may be calculated and defined by arithmetic operations using multiple evaluation values, calculated and defined by combining multiple evaluation values ​​and multiple arithmetic operations, or defined by setting a function for calculating priority and substituting the value into it. Evaluation values ​​include, for example, impact, frequency, proximity, similarity, density, and risk. Impact indicates the degree to which a process performed based on real-world information affects the overall presentation in the virtual space. Frequency indicates the degree to which a process is performed. Proximity indicates the degree to which an object or coordinate is close. Similarity indicates the degree to which the shapes and colors of multiple objects are similar. Density indicates how many other objects are present within a certain range of an object. Risk indicates the possibility of interfering with the operation of the system or other objects. As an example, the impact coefficient is defined by using impact and frequency as multiple evaluation values, and is calculated as the product of impact and frequency. If the impact or frequency changes, the impact coefficient (i.e., the priority of the process) may change through recalculation. In this way, by determining the impact coefficient (priority) using multiple evaluation values, flexible prioritization can be achieved.

[0076] Now, with reference to Figure 9, priority information in a modified example of this embodiment will be described. Figure 9 is a diagram showing an example of priority information in a modified example of this embodiment. As shown in Figure 9, priority information is associated with real-world spatial information, impact level, frequency, and priority (impact coefficient), and may also be associated with the corresponding processing. For example, the first piece of "location information" acquired as object information within the real-world space information is associated with an influence level of "0.3", a frequency of "0.9", an impact coefficient of "0.27", and the processing "object alignment". The "lamp color" acquired as environmental information within the real-world space information is associated with an influence level of "1.0", a frequency of "0.3", an impact coefficient of "0.3", and the processing "scene transition". The second piece of "location information" is associated with an influence level of "0.5", a frequency of "0.5", an impact coefficient of "0.25", and the processing "animation playback". Assuming that the priority information shown in Figure 9 is used, and both environmental information indicating the "lamp color" and object information indicating the "position" are obtained as real-world spatial information, the processing will be executed in the following order: "scene transition," "object alignment," and "animation playback."

[0077] Modified examples of this embodiment have been described above. Furthermore, some or all of the functions of the information processing system 1, the real object 10, and the information processing device 30 in the above-described embodiment may be implemented by a computer. In that case, the functions may be implemented by recording a program for implementing these functions on a computer-readable recording medium, and then having the computer system read and execute the program recorded on this recording medium. In this context, "computer system" includes hardware such as the operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Additionally, "computer-readable recording media" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs over networks such as the Internet or communication lines such as telephone lines, as well as those that hold programs for a fixed period of time, such as volatile memory within computer systems that act as servers or clients in such cases. Furthermore, the above program may be for the purpose of implementing some of the functions described above, or it may be for the purpose of implementing the above functions in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0078] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the spirit of this invention. [Explanation of Symbols]

[0079] 1...Information processing system, 10 (10-1 to 10-n)...Real object, 11 (11-1 to 11-3)...Virtual object, 20...Sensor device, 30...Information processing device, 40...Display, 110...Communication unit, 120...Drive unit, 130...Sensor unit, 140...Control unit, 141...State detection unit, 310...Communication unit, 320...Input unit, 330...Storage unit, 340...Control unit, 341...Digital space management unit, 342...Real space information acquisition unit, 343...Digital space information acquisition unit, 344...Priority processing unit, 345...Processing order control unit, 346...Operation control unit, 347...Display control unit, 350...Output unit

Claims

1. A real-space information acquisition unit acquires real-space information indicating that the state of the real space represented by a model in which a first object is placed in the real world has changed, A priority processing unit that determines the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information, A processing order control unit sorts the execution order of processes performed in a digital space that reproduces the real space by placing a second object corresponding to the first object, based on the real space information, in order of priority based on the priority determination result. A digital space management unit that executes the process in the sorted execution order, An information processing system equipped with the following features.

2. The real-space information includes at least one of environmental information indicating that the environment of the real-space has changed, and object information indicating that the state of the first object placed in the real-space has changed. The information processing system according to claim 1.

3. When the aforementioned environmental information and the aforementioned object information are acquired, if the processing performed based on the environmental information is given priority, The real-space information acquisition unit acquires the object information again after the execution of the process, The digital space management unit then uses the object information that has been acquired again to execute the processing that is performed based on the object information. The information processing system according to claim 2.

4. The aforementioned priority is a value defined by a function of at least one evaluation value. The information processing system according to claim 1.

5. The aforementioned priority is a value calculated by multiplying the degree of influence, which indicates the extent to which the processing performed based on the real-world information affects the overall presentation in the digital space, by the frequency, which indicates the extent to which the processing is performed. The information processing system according to claim 4.

6. A digital space information acquisition unit acquires digital space information indicating that the state of the second object has changed as a result of the execution of the process in the digital space. An operation control unit that controls the operation of the first object corresponding to the second object based on the digital space information, The information processing system according to claim 1, comprising:

7. A process for acquiring real-space information that obtains real-space information indicating that the state of the real space represented by a model in which the first object is placed in the real world has changed, A priority processing process that determines the priority of the acquired real-space information based on priority information indicating the priority set for each piece of real-space information, A processing order control process that sorts the execution order of processes performed based on the real-world information in a digital space that reproduces the real-world space by placing a second object corresponding to the first object, in order of priority based on the priority determination result, A digital space management process that executes the processes in the sorted execution order, A computer-based information processing method that includes [a specific component / function].

8. Computers A means for acquiring real-space information that obtains real-space information indicating that the state of the real space represented by a model in which a first object is placed in the real world has changed, Priority processing means for determining the priority of acquired real-space information based on priority information indicating the priority set for each piece of real-space information, Processing order control means for sorting the execution order of processes performed based on the real-world information in a digital space that reproduces the real-world space by arranging a second object corresponding to the first object, in order of priority based on the priority determination result, A digital space management means that executes the process in the sorted execution order, A program designed to function as such.

Images