System, information processing method and program

Abstract

To enable efficiently grasping a real space when generating a virtual space obtained by virtualizing the real space.SOLUTION: A system includes: a management section for managing a first area where a state of a real space is reflected and a second area where the state of the real space is not reflected in a virtual space obtained by virtualizing the real space based on information related to the state of the real space; an estimation section for estimating a third area where the information related to the state of the real space is required on a movement route in the real space of a first mobile body; a detection section for detecting a fourth area where the third area and the second area overlap each other; and an instruction section for instructing a second mobile body to acquire the state of the real space corresponding to the fourth area.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a system, an information processing method, and a program. [Background technology]

[0002] Currently, a technology called digital twin, which reproduces real space in a virtual space, is being considered. In order to realize a digital twin, it is necessary to digitize the real space using a large number of sensors. For example, Patent Document 1 discloses a distributed discrete system that uses a digital twin. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-161342 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, if we were to cover the entire real space with a virtual space, there would be a problem that it would be impractical because it would require a huge number of sensors. [Means for solving the problem]

[0005] A system according to one embodiment of the present invention includes a management unit that manages a first area in which the state of real space is reflected and a second area in which the state of real space is not reflected in a virtual space that is a virtualized version of real space based on information regarding the state of the real space; an estimation unit that estimates a third area on a movement path in the real space of a first moving body that requires information regarding the state of the real space; a detection unit that detects a fourth area where the third area and the second area overlap; and an instruction unit that instructs a second moving body to obtain the state of the real space corresponding to the fourth area. Effect of the Invention

[0006] According to the present invention, it is possible to provide a technique that makes it possible to suppress the divergence between the fourth area and the real space when generating a virtual space by virtualizing the real space. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a management system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining a state of a virtual space in which a state of a real space is reflected. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional block configuration of an information processing device. [Diagram 5] 13 is a flowchart showing an example of a processing procedure for generating and using a virtual space. [Figure 6] 13 is a flowchart illustrating an example of a processing procedure for updating a cover area. [Figure 7] FIG. 11 is a diagram for explaining a specific example when updating a cover area. [Figure 8] FIG. 11 is a diagram for explaining a specific example when updating a cover area. [Figure 9] FIG. 11 is a diagram for explaining a specific example when updating a cover area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described with reference to the accompanying drawings, in which the same reference numerals denote the same or similar configurations.

[0009] 1 is a diagram showing an example of a management system 1 according to the present embodiment. The management system 1 includes an information processing device 10, a vehicle 20, an aircraft 30, and a fixed sensor 40.

[0010] The information processing device 10 reproduces the real space in the virtual space based on information on the state of the real space received from a plurality of vehicles 20 traveling in the real space, an aircraft 30 flying in the real space, and a fixed sensor 40 installed in the real space. Reproducing the real space in the virtual space is equivalent to copying the real space in the virtual space, and is therefore called a "digital twin". The virtual space V in FIG. 1 shows the state of the digital twin reproduced by the information processing device 10 based on information on the state of the real space collected by the vehicle 20, the aircraft 30, and the fixed sensor 40. In the example of FIG. 1, the state on the ground is illustrated, but the real space reproduced in the virtual space by the information processing device 10 is not limited to the ground, and may also include the state of the sky (for example, up to an altitude of several hundred meters from the ground).

[0011] The vehicle 20 is assumed to be an automobile, truck, bus, etc. that runs on the ground, but is not limited thereto. The vehicle 20 may also include a train, a motorcycle, a bicycle, etc. The vehicle 20 includes a vehicle equipped with a sensor and a vehicle not equipped with a sensor. The vehicle 20 equipped with a sensor can sense the state of the real space around the vehicle 20. In addition, the vehicle 20 equipped with a sensor transmits information related to the state of the real space obtained by sensing to the information processing device 10 together with position information indicating the current position of the vehicle 20.

[0012] The flying object 30 is equipped with a sensor and can sense the state of real space while flying in the air. The flying object 30 transmits information about the state of real space obtained by sensing to the information processing device 10 together with position information indicating the current position of the flying object 30. The flying object 30 may be, for example, a remotely controlled drone or a small helicopter. The flying object 30 may also be mounted on the vehicle 20.

[0013] The fixed sensor 40 senses the state of the real space around itself. The fixed sensor 40 may be placed in a place with many blind spots, such as an intersection. The fixed sensor 40 transmits information about the state of the real space obtained by sensing to the information processing device 10 together with position information indicating the installation location of the fixed sensor 40.

[0014] The sensors provided in the vehicle 20, the flying object 30, and the fixed sensor 40 may be, for example, a camera, a LiDAR (Light Detection And Ranging), a microphone, etc., but are not limited to these. Any sensor may be used as long as it can acquire the state of the real space.

[0015] The information on the state of the real space transmitted from the vehicle 20, the flying object 30, and the fixed sensor 40 to the information processing device 10 includes information indicating what object exists at what location and when. For example, the information on the state of the real space may be data combining still images or moving images captured by a camera with position information, or data combining point cloud data detected by laser scanning with LiDAR with position information.

[0016] The information processing device 10 manages a virtual space that reflects the state of the real space, based on information about the state of the real space acquired from the vehicle 20, the aircraft 30, and the fixed sensor 40. More specifically, the information processing device 10 manages an area in the virtual space where the state of the real space is reflected (hereinafter referred to as a "covered area" or "first area") and an area in which the state of the real space is not reflected (hereinafter referred to as a "non-covered area" or "second area").

[0017] In this embodiment, a moving body that moves in real space and moves by using information of a virtual space that reflects the state of the real space is referred to as an "information using moving body" or a "first moving body". In addition, the information processing device 10 can support the movement of the information using moving body by transmitting information of the virtual space that reflects the state of the real space to the information using moving body. The information using moving body may be, for example, a vehicle 20 with an automatic driving function, and an aircraft 30 with an automatic flight function.

[0018] FIG. 2 is a diagram for explaining the state of the virtual space in which the state of the real space is reflected. The area represented by the mesh indicates the covered area CA, and the other areas are non-covered areas NCA. In addition, the vehicles 20-1 to 20-4 shown in FIG. 2 are equipped with sensors, and acquire information on the state of the real space in a predetermined range centered on the vehicle 20 and transmit it to the information processing device 10. In addition, the fixed sensor 40 also acquires information on the state of the real space in a predetermined range centered on the fixed sensor 40 and transmits it to the information processing device 10. However, the objects B1 and B2 are obstacles, and the sensor mounted on the vehicle 20 and the fixed sensor 40 cannot acquire information on the state of the real space on the back side of the objects B1 and B2. For example, the object B1 exists in front of the vehicle 20-4, but the sensor of the vehicle 20-4 cannot acquire information on the back side of the object B1, and the area is not included in the covered area CA of the other vehicles 20, so the back side of the object B1 is a non-covered area NCA.

[0019] Here, in order to manage the virtual space for the entire range of the real space, it is necessary to place a huge number of vehicles 20, flying objects 30, and fixed sensors 40 in the real space, which is not realistic. Therefore, the information processing device 10 estimates an area (hereinafter referred to as a "demand area" or "third area") on the moving path of the information using mobile object moving through the real space, where information on the state of the real space is required. In addition, an area where the demand area and the non-covered area overlap is defined as an area where information on the state of the real space is insufficient (hereinafter referred to as an "information insufficient area" or "fourth area"), and the information processing device 10 instructs a predetermined mobile object (hereinafter referred to as an "information acquiring mobile object" or "second mobile object") to acquire information on the state of the real space in the information insufficient area. The information acquiring mobile object may be, for example, a dedicated flying object 30 prepared for collecting information on the state of the real space in the information insufficient area. In addition, the information processing device 10 updates the information insufficient area so that it becomes a covered area, using information on the state of the real space received from the information acquiring mobile object.

[0020] In the example of Fig. 2, vehicle 20-1, vehicle 20-2, and vehicle 20-4 are vehicles 20 equipped with an automatic driving function, and areas DA1, DA2, and DA4 correspond to demand areas of vehicle 20-1, vehicle 20-2, and vehicle 20-4, respectively. Note that in the example of Fig. 2, for convenience of explanation, the range of the demand area is shown as a sector, but the range of the demand area is not limited to a sector shape.

[0021] <Hardware configuration> 3 is a diagram showing an example of a hardware configuration of the information processing device 10. The information processing device 10 has a processor 11 such as a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), a memory (e.g., RAM or ROM), a storage device 12 such as an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a network IF (Network Interface) 13 for wired or wireless communication, an input device 14 for accepting input operations, and an output device 15 for outputting information. The input device 14 is, for example, a keyboard, a touch panel, a mouse, and / or a microphone. The output device 15 is, for example, a display, a touch panel, and / or a speaker.

[0022] The information processing device 10 may be configured with one or more physical servers, may be configured with a virtual server operating on a hypervisor, or may be configured with a cloud server. The information processing device 10 that executes the process according to the present embodiment may be called a "system." Note that the term "system" does not necessarily mean that the information processing device 10 that executes the process according to the present embodiment is a plurality of hardware devices (including physical servers and virtual servers), but also includes a case where the information processing device 10 is a single piece of hardware.

[0023] <Function block configuration> FIG. 4 is a diagram showing an example of a functional block configuration of the information processing device 10. The information processing device 10 includes a storage unit 100, a management unit 101, an estimation unit 102, a detection unit 103, an instruction unit 104, a calculation unit 105, and a transmission unit 106. The storage unit 100 can be realized by using a storage device 12 included in the information processing device 10. The management unit 101, the estimation unit 102, the detection unit 103, the instruction unit 104, and the calculation unit 105 can be realized by the processor 11 of the information processing device 10 executing a program stored in the storage device 12. The program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a Universal Serial Bus (USB) memory or a Compact Disc Read-Only Memory (CD-ROM).

[0024] The storage unit 100 stores a virtual space DB 100a. The virtual space DB 100a stores various data necessary for reproducing the real world in a virtual space. For example, the virtual space DB 100a may include data such as the driving positions, driving directions, and vehicle speeds of a plurality of vehicles 20 driving in the real space, data such as the flying positions, flying directions, and flying speeds of a plurality of flying objects 30 flying in the real space, road data, position and shape data of buildings, structures, traffic lights, road signs, and the like that exist in the real space, position and shape data of people that exist in the real space, and the like.

[0025] The management unit 101 manages a covered area (first area) in which the state of the real space is reflected and a non-covered area (second area) in which the state of the real space is not reflected, in a virtual space obtained by virtualizing the real space, based on information on the state of the real space acquired from the vehicle 20, the aircraft 30, the fixed sensor 40, etc. Note that areas other than the covered areas in the virtual space managed by the information processing device 10 correspond to non-covered areas.

[0026] The estimation unit 102 estimates a demand area (third area) on the movement path in real space of the information using mobile body that requires information on the state of the real space. The estimation unit 102 may estimate the demand area based on a pre-setting regarding the course along which the information using mobile body moves and the range of the demand area at the current position of the information using mobile body (for example, a sector-shaped range in the direction of travel as shown in FIG. 2). The pre-setting may be, for example, setting information of a car navigation system. The estimation unit 102 may also estimate a course estimated from the movement speed and movement direction of the information using mobile body, and estimate the demand area based on the estimated course and the range of the demand area at the current position of the information using mobile body.

[0027] The detection unit 103 detects an information-deficient area (fourth area) where a demand area (third area) and a non-coverage area (second area) overlap.

[0028] The instruction unit 104 instructs the information acquisition mobile body (second mobile body) to acquire a state of real space corresponding to the information insufficient area (fourth area) detected by the detection unit 103. The instruction unit 104 may also instruct an information acquisition mobile body, among the multiple information acquisition mobile bodies (second mobile bodies), that is located closest to the information insufficient area (fourth area) to acquire information about the state of real space corresponding to the information insufficient area. The instruction unit 104 may also instruct an information acquisition mobile body, among the multiple information acquisition mobile bodies (second mobile bodies), that is located closest to the information insufficient area (fourth area) and whose battery charge amount is equal to or greater than a predetermined rate, to acquire information about the state of real space corresponding to the information insufficient area.

[0029] The information acquiring mobile object acquires the state of the real space corresponding to the information-deficient area according to the instruction and transmits it to the information processing device 10. The management unit 101 also updates the cover area based on the information on the state of the real space acquired from the information acquiring mobile object (i.e., the area that was an information-deficient area is updated to a cover area).

[0030] In addition, when the information-acquiring mobile body is mounted on a vehicle, the instruction unit 104 may instruct the vehicle mounting the information-acquiring mobile body (second mobile body) to acquire the state of real space corresponding to the information-deficient area (fourth area).

[0031] The calculation unit 105 calculates the time required for the information acquiring mobile object (second mobile object) to move to the information lacking area (fourth area) based on the current position of the information acquiring mobile object.

[0032] The transmitting unit 106 transmits information about the virtual space within the cover area (first area) to the information using mobile object.

[0033] <Processing Procedure> FIG. 5 is a flowchart showing an example of a processing procedure for generating and using a virtual space.

[0034] In step S100, the management unit 101 collects information on the state of the real space from the vehicle 20, the flying object 30, and the fixed sensor 40. The vehicle 20, the flying object 30, and the fixed sensor 40 transmit information on the state of the real space to the information processing device 10 at a predetermined period (for example, every second, every 10 seconds, every 30 seconds, every minute, etc.) in association with position information indicating the current position and a time stamp. In addition, when the information on the state of the real space is a camera image, the information on the state of the real space also includes information on the shooting direction. The predetermined period may be variable according to the degree of change in the surrounding state of the vehicle 20, the flying object 30, and the fixed sensor 40. For example, in the case of a traveling vehicle 20, since the surrounding state changes from moment to moment, the vehicle 20 may transmit information on the state of the real space to the information processing device 10 at a high frequency (for example, every second, etc.). On the other hand, when the vehicle 20 is installed in the same place as the fixed sensor 40 and the surrounding state does not change, the fixed sensor 40 may transmit information on the state of the real space to the information processing device 10 at a low frequency (for example, every minute, etc.).

[0035] In addition, the management unit 101 periodically acquires data such as the driving position, driving direction, and vehicle speed from the vehicle 20 and the flying object 30, as well as data such as the current position, flight direction, and flight position, and stores the data in the virtual space DB 100a.

[0036] In step S101, the management unit 101 reproduces the real space in the virtual space based on the collected information on the state of the real space. For example, the management unit 101 detects the type, size, and position (absolute position) of an object existing in the real space by analyzing image data and / or point cloud data transmitted from the vehicle 20, the aircraft 30, and the fixed sensor 40 as information on the state of the real space, and stores the detection results in the virtual space DB 100a.

[0037] Furthermore, the management unit 101 detects the range of the covered area and the range of the non-covered area by analyzing the image data and / or point cloud data transmitted from the vehicle 20, the flying object 30, and the fixed sensor 40, and stores the detection results in the virtual space DB 100a. The detection process of the size of an object, the position of an object, the range of the covered area, and the range of the non-covered area may be realized by using an existing library, etc. As a result, for example, as shown in FIG. 2, the range of the covered area and the range of the non-covered area are reflected in the virtual space, and the position, size, etc. of an object existing in the covered area are also reflected in the virtual space.

[0038] Furthermore, the detection unit 103 detects an information-deficient area where a demand area and a non-coverage area overlap, and the instruction unit 104 instructs the information-acquiring mobile object to acquire the state of real space corresponding to the detected information-deficient area. Furthermore, the management unit 101 analyzes information on the state of real space acquired from the information-acquiring mobile object, and detects the type, size, and position (absolute position) of an object present in the information-deficient area, thereby updating the information-deficient area to a covered area.

[0039] Specifically, the estimation unit 102 estimates a demand area (third area) on the moving route in the real space of the information using mobile body, where information on the state of the real space is required. The estimation unit 102 may estimate the demand area along the moving route based on a pre-setting regarding the moving route along which the information using mobile body moves and the range of the demand area at the current position of the information using mobile body (for example, a sector-shaped range in the traveling direction as shown in FIG. 2). The pre-setting may be, for example, setting information of a car navigation system. The estimation unit 102 may also estimate the moving route from the moving speed and moving direction of the information using mobile body, and estimate the demand area along the moving route based on the estimated moving route and the range of the demand area at the current position of the information using mobile body. Next, the detection unit 103 detects an information-deficient area (fourth area) where the demand area (third area) and the non-covered area (second area) overlap. Next, the instruction unit 104 instructs the information acquisition mobile object (second mobile object) to acquire the state of real space corresponding to the information-deficient area (fourth area) detected by the detection unit 103. Furthermore, the management unit 101 acquires the state of real space corresponding to the information-deficient area (fourth area) and updates the information-deficient area to a cover area.

[0040] In step S102, the transmitting unit 106 transmits information on the virtual space that reproduces the real space in the covered area to the information using mobile body. For example, the transmitting unit 106 may transmit information on the virtual space within a predetermined range from the current position of the information using mobile body to the information using mobile body. The information on the virtual space may include information for identifying the covered area and the non-covered area in the virtual space, the position of an object existing in the covered area, and the like. The information on the virtual space may also include information that is estimated to occur in the virtual space in the future and is related to the information using mobile body (for example, information that another mobile body will appear on the traveling path of the information using mobile body in a few seconds, etc.). The processing procedure of step S102 is not essential. The information processing device 10 may not perform a process of transmitting the information on the virtual space to the information using mobile body.

[0041] In step S103, if the information processing device 10 is instructed to end the processing related to the virtual space, the information processing device 10 ends the processing, and if the information processing device 10 has not received an instruction to end the processing related to the virtual space, the information processing device 10 repeats the processing procedure from step S100 to step S103. By repeating the processing procedure from step S100 to step S103, the information on the virtual space is always updated to the latest state. For example, when the vehicle 20 moves, the information on the virtual space generated based on the information on the state of the real space acquired before the movement is updated based on the information on the state of the real space acquired after the movement.

[0042] Fig. 6 is a flowchart showing an example of a processing procedure for updating a cover area. The processing procedure in Fig. 6 corresponds to the processing procedure in step S101 in Fig. 5, in which the management unit 101 updates an information-deficient area to a cover area.

[0043] In the following description, the information using mobile body is a vehicle 20 equipped with an automatic driving function (hereinafter referred to as "automatically driven vehicle 20A"). The information acquiring mobile body is an air vehicle 30. The information processing device 10 is also assumed to know the current position of the air vehicle 30 that can be used as an information acquiring mobile body. The automatically driven vehicle 20A is also assumed to be equipped with a camera or LiDAR, etc., and to be capable of acquiring information regarding the state of the real space around it.

[0044] In the processing procedure of FIG. 6, when an information-deficient area appears at a time (t seconds later) in the future than the current time, the information processing device 10 updates the information-deficient area to a covered area by the future time by having the flying object 30 obtain information on the state of the real space ahead of the future time. That is, when the autonomously driven vehicle 20A (target vehicle) travels from the current position of the autonomously driven vehicle 20A along a travel route, the detection unit 103 calculates the time from the current time until the information-deficient area (fourth area) appears, and when the time required for the flying object 30 to move from the current position to the information-deficient area is shorter than the time from the current time until the information-deficient area appears, the instruction unit 104 instructs the flying object 30 to obtain the state of the real space corresponding to the information-deficient area. A specific description will be given below.

[0045] In step S200, the estimation unit 102 predicts a travel route from 0 seconds to X seconds later for each autonomous vehicle 20A (first moving body) moving through real space, assuming that the current time is 0 seconds. For example, it is considered that the autonomous vehicle 20A often has a travel route from a departure point to a destination set in advance, like a car navigation system. In this regard, the estimation unit 102 may estimate a travel route from 0 seconds to X seconds later for the autonomous vehicle 20A by matching the previously set travel route with information indicating the current position of the autonomous vehicle 20A. The value of X seconds later is arbitrary, and may be set in advance to, for example, 10 seconds later, 20 seconds later, 30 seconds later, or the like.

[0046] In step S201, the estimation unit 102 predicts the vehicle position of each autonomous vehicle 20A from 0 seconds to X seconds later at intervals of d seconds based on the travel route predicted in step S200. d seconds may be any value smaller than X seconds, and may be set to, for example, 1 second, 2 seconds, etc.

[0047] In step S202, the estimation unit 102 predicts the covered areas, uncovered areas, and demand areas from 0 seconds to X seconds later at d second intervals, based on the vehicle positions of each autonomously driven vehicle 20A predicted in step S101.

[0048] In step S203, the estimation unit 102 sets the value of a counter t to zero.

[0049] In step S204, the estimation unit 102 estimates whether or not there is an information-insufficient area after t seconds. Specifically, the estimation unit 102 determines whether or not there is an area where the non-covered area and the demand area overlap (i.e., an information-insufficient area) for the covered area, non-covered area, and demand area after t seconds, among the covered area, non-covered area, and demand area estimated in the processing procedure of step S202 at d-second intervals. If there is an information-insufficient area, the process proceeds to the processing procedure of step S205, and if there is no information-insufficient area, the process proceeds to the processing procedure of step S207.

[0050] In step S205, the estimation unit 102 determines whether or not there is an aircraft 30 that can move to the information-deficient area within t seconds. If there is, the process proceeds to step S206, and if there is not, the process proceeds to step S207. Note that, if there are multiple aircraft 30 that can move to the information-deficient area within t seconds, the instruction unit 104 may instruct the aircraft 30 that is closest to the information-deficient area (fourth area) among the multiple aircraft 30 (second moving bodies) to acquire information regarding the state of the real space corresponding to the information-deficient area.

[0051] In step S206, the instruction unit 104 instructs the flying object 30 that can move to the information-deficient area within t seconds to obtain information about the state of real space for the information-deficient area determined in the processing procedure of step S204.

[0052] For example, the instruction unit 104 may notify the aircraft 30 of coordinate information indicating the shooting range (i.e., the range of the information-shortage area) and may instruct the aircraft 30 to acquire information regarding the state of the real space. When the information regarding the state of the real space is acquired from the aircraft 30, the management unit 101 updates the cover area based on the information regarding the state of the real space acquired from the aircraft 30.

[0053] In step S207, the estimation unit 102 updates the value of t to t+d, where d is the same as the value of d seconds used in steps S201 and S202. For example, when t=0 and the value of d seconds used in steps S201 and S202 is 1 second, the estimation unit 102 updates t to 1.

[0054] In step S208, the estimation unit 102 determines whether the value of t is equal to or less than X. Note that X is the same as the value of X seconds used in steps S200 to S202. If the value of t is equal to or less than X, the process returns to step S204, and if the value of t exceeds X, the process ends.

[0055] The information processing device 10 repeatedly executes the process procedure of steps S200 to 208 at intervals of X seconds or less. At this time, the predetermined interval may be, for example, X / 2 seconds.

[0056] In addition, the information processing device 10 may remember that it has instructed the flying object 30 to obtain information regarding the state of real space in the information-deficient area in the processing procedure of step S206, and may avoid issuing duplicate instructions to the flying object 30 in the next cycle.

[0057] In the processing procedure of step S205, it is assumed that there is a low possibility that a flying object 30 capable of moving into the information-insufficient area will exist 0 seconds or 1 second after the current time. Therefore, the information processing device 10 may perform prediction from a predetermined time (e.g., 5 seconds or 10 seconds) to X seconds after the processing procedure of steps S200 to S202, instead of performing prediction from 0 seconds to X seconds after. In this case, the processing procedure of step S203 also needs to be changed to "t = predetermined time (e.g., 5 seconds or 10 seconds)".

[0058] 7 to 9 are diagrams for explaining a specific example when updating the cover area. FIG. P100 shows the state of the real space at the time t=0. In FIG. P100, vehicles 20-5 and 20-6 are traveling on a road. Furthermore, vehicle 20-5 is an autonomous vehicle and is equipped with a camera or the like for capturing images of its surroundings. On the other hand, vehicle 20-6 is not an autonomous vehicle, and vehicle 20-6 is not equipped with a camera or the like for capturing images of its surroundings. Note that information processing device 10 is capable of communicating with car navigation devices mounted on vehicles 20-5 and 20-6, and is capable of acquiring travel routes of vehicles 20-5 and 20-6.

[0059] In FIG. P100, a demand area DA exists inside a coverage area CA. Therefore, unless there is an obstacle in front of the vehicle 20-5, it is assumed that the demand area D will not become a part of the coverage area CA.

[0060] FIG. P101 shows a state where d seconds have passed since FIG. P100, and shows a state where the vehicle 20-6 is approaching the demand area DA. FIG. P102 shows a state where d seconds have passed since FIG. P101. In FIG. P102, the vehicle 20-6 is present ahead of the vehicle 20-5, so the part of the demand area DA that is shaded by the vehicle 20-6 as seen from the vehicle 20-5 becomes the non-covered area NCA. In other words, assuming that the current time is t=0, the non-covered area NCA will appear after 2d seconds. FIG. P103 shows a state where d seconds have passed since FIG. P102. In FIG. P103, the vehicle 20-6 is not present ahead of the vehicle 20-5, so the non-covered area NCA has disappeared.

[0061] 7 and 8, the estimation unit 102 estimates that the information-deficient area NCA will appear 2d seconds after the current time. Next, the estimation unit 102 determines whether or not there is an aircraft 30 that can move into the information-deficient area NCA within 2d seconds after the current time. Here, it is assumed that the aircraft 30 located at the charging station can move into the information-deficient area NCA within 2d seconds.

[0062] Next, the instruction unit 104 instructs the flying object 30 located at the charging station to move to the information deficient area NCA.

[0063] Using Fig. 9, we will explain how the flying object 30 moves to an information-deficient area. Fig. P104, like Fig. P100, shows the state of real space at time t=0. Fig. P105, like Fig. P101, shows the state of real space after d seconds have passed. However, as explained in Figs. 7 and 8, the estimation unit 102 has already estimated that a non-coverage area NCA will appear after 2d seconds.

[0064] FIG. P105 shows how the flying object 30, which has been instructed to move to the information-deficient area NCA, moves ahead and into the airspace above the information-deficient area NCA by d seconds later. The flying object 30 moves to the airspace above the information-deficient area NCA, acquires information about the state of the real space, and transmits the acquired information to the information processing device 10. In this way, when the vehicle 20-5 turns right at the intersection, the flying object 30 acquires information about the state of the real space in the part that is in the shadow of the vehicle 20-6 coming from the front (i.e., the part of the information-deficient area NCA) ahead, so that the vehicle 20-5 can continue autonomous driving without the information-deficient area NCA appearing.

[0065] <Summary> According to the embodiment described above, the information processing device 10 estimates the covered area and the uncovered area in the virtual space based on the information on the state of the real space acquired from the vehicle 20, the flying object 30, the fixed sensor 40, etc., and when it is estimated that an information-deficient area will exist in the future, it instructs the information-using mobile object to acquire the state of the real space corresponding to the information-deficient area. This makes it possible to efficiently grasp the real space when generating a virtual space that virtualizes the real space. In addition, when generating a virtual space that virtualizes the real space, it is possible to express to what extent the system can grasp which places in the real space, and to use this to suppress the divergence between the real space and the digital twin.

[0066] Furthermore, according to this embodiment, by acquiring the state of the real space corresponding to the information-insufficient area, it is possible to efficiently realize a virtual space (i.e., a digital twin) with higher accuracy than before. When a highly accurate virtual space is realized in this way, it becomes possible to realize various things in the virtual space that could not be realized by conventional simulation technology, such as realizing a highly accurate automatic driving simulation required for the development of automatic driving technology in the virtual space, and collecting data in the virtual space assuming that automatic driving is performed in the real space. In addition, since it becomes possible to perform things that have been performed using the real space up to now in the virtual space, it is also possible to dramatically improve the speed and efficiency of technological development. Furthermore, the virtual space realized by this embodiment can be used not only in the field of automatic driving, but also in technological development in various fields.

[0067] According to this embodiment, the real space can be grasped efficiently in the digital twin. Therefore, the technology according to this embodiment can contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and infrastructure." [Explanation of symbols]

[0068] 1... management system, 10... information processing device, 11... processor, 12... storage device, 13... network IF, 14... input device, 15... output device, 20... vehicle, 30... flying object, 40... fixed sensor, 100... storage unit, 100a... virtual space DB, 101... management unit, 102... estimation unit, 103... detection unit, 104... instruction unit, 105... calculation unit, 106... transmission unit

Claims

1. A management unit that manages, based on information regarding a real-time state of a real space including dynamic information indicating the size and position of an object, a first area in which the dynamic information contained in the real-time state of the real space is reflected and a second area in which the dynamic information contained in the real-time state of the real space is not reflected, in a virtual space that virtualizes the real space; an estimation unit that estimates a third area, which is a predetermined range including a traveling direction of the first moving body on a moving path of the first moving body in the real space, and indicates the predetermined range after a predetermined time when the first moving body proceeds along the moving path; a detection unit that detects a fourth area where the third area and the second area overlap after the predetermined time; an instruction unit that instructs a second moving object capable of moving to the fourth area within the predetermined time period to acquire a real-time state of the real space corresponding to the fourth area; A system having

2. the management unit updates the first area by reflecting dynamic information included in information regarding a real-time state of the real space corresponding to the fourth area, which is acquired from the second moving object, in the first area. The system of claim 1 .

3. the instruction unit instructs one of a plurality of second moving bodies that is located closest to the fourth area to acquire a state of the real space corresponding to the fourth area. The system of claim 1 .

4. a calculation unit that calculates a required time required for the second moving object to move to the fourth area based on a current position of the second moving object, the instruction unit instructs the second moving object to acquire a real-time state of the real space corresponding to the fourth area when the required time is shorter than the predetermined time. The system of claim 1 .

5. The second moving body is mounted on a vehicle. The system of claim 1 .

6. The first moving body is a vehicle that travels on the ground, and the second moving body is an aircraft that flies in the air. A system according to any one of claims 1 to 5.

7. An information processing method executed by an information processing device, a step of managing a first area in a virtual space obtained by virtualizing a real space based on real-time information regarding a state of the real space including dynamic information indicating a size and a position of an object, the first area reflecting the dynamic information included in the real-time state of the real space and a second area in which the dynamic information included in the real-time state of the real space is not reflected, the virtual space being obtained by virtualizing a real space; estimating a third area, which is a predetermined range including a traveling direction of the first moving body on a moving path of the first moving body in the real space, and which indicates the predetermined range after a predetermined time when the first moving body proceeds along the moving path; detecting a fourth area where the third area and the second area overlap after the predetermined time; instructing a second moving object capable of moving to the fourth area within the predetermined time period to acquire a real-time state of the real space corresponding to the fourth area; An information processing method comprising:

8. On the computer, a step of managing a first area in a virtual space obtained by virtualizing a real space based on real-time information regarding a state of the real space including dynamic information indicating a size and a position of an object, the first area reflecting the dynamic information included in the real-time state of the real space and a second area in which the dynamic information included in the real-time state of the real space is not reflected, the virtual space being obtained by virtualizing a real space; estimating a third area, which is a predetermined range including a traveling direction of the first moving body on a moving path of the first moving body in the real space, and which indicates the predetermined range after a predetermined time when the first moving body proceeds along the moving path; detecting a fourth area where the third area and the second area overlap after the predetermined time; instructing a second moving object capable of moving to the fourth area within the predetermined time period to acquire a real-time state of the real space corresponding to the fourth area; A program for executing the above.

Images