3D remote sensing system, unmanned mobile device control system, unmanned mobile device, unmanned mobile device for markers, unmanned mobile device swarm control system, control circuit, storage medium, 3D remote sensing system operation method, unmanned mobile device control system operation method, and unmanned mobile device operation method

Abstract

To provide a three-dimensional remote sensing system (100) capable of synthesizing three-dimensional data of an object using imaging data captured by an unmanned mobile body (40), the system comprises an unmanned mobile body control device (10) that generates an observation plan including at least a movement plan for the unmanned mobile body (40), an unmanned mobile body (40) that images an object using an imaging sensor (43) based on the observation plan and generates mobile body data including the imaging data and imaging conditions, and a server (50) that synthesizes three-dimensional data that represents the object in three dimensions using the mobile body data.

Description

[Technical Field]

[0001] The present disclosure relates to a three-dimensional remote sensing system that synthesizes three-dimensional data, an unmanned mobile object control device, an unmanned mobile object, an unmanned mobile object for marker use, an unmanned mobile object for positioning, an unmanned mobile object for communication, a server, an unmanned mobile object group control system, a control circuit, a storage medium, a three-dimensional remote sensing system operation method, an unmanned mobile object control device operation method, an unmanned mobile object operation method, and a server operation method. [Background technology]

[0002] Conventionally, inspecting power lines and the like installed on steel towers required workers to climb the towers to inspect them, which required a great deal of time and effort. In recent years, for such high-altitude structures, unmanned aircraft such as drones have been used to photograph the target structure and inspect the target structure using the captured image. For example, Patent Document 1 discloses a technology in which an unmanned aircraft is equipped with a camera for photographing the target object, and image data captured by the camera is used for inspecting elevated power lines, surveying terrain, and the like. In Patent Document 1, even if the unmanned aircraft fails to photograph the target object, the unmanned aircraft re-photographs the object from the position where the photographing failure was detected, thereby eliminating the hassle of flying the unmanned aircraft along the same route to re-photograph the object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-191523 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the above-mentioned conventional technology, image data obtained by a photographic camera is used directly for inspection, surveying, etc. While this is a good method of use in normal times, for example, when a disaster occurs, 3D (Three Dimensions) data, i.e., three-dimensional data, of the object is required to accurately grasp the condition of the object damaged by the disaster from a remote location. However, the above-mentioned conventional technology has a problem in that it is not possible to obtain three-dimensional data of the object from image data.

[0005] The present disclosure has been made in consideration of the above, and aims to provide a three-dimensional remote sensing system that can synthesize three-dimensional data of an object using imaging data captured by an unmanned moving body. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the three-dimensional remote sensing system of the present disclosure includes an unmanned mobile body control device that generates an observation plan including at least a movement plan for the unmanned mobile body, an unmanned mobile body that captures images of an object using an imaging sensor based on the observation plan and generates mobile body data including the image data and the imaging conditions, and a server that uses the mobile body data to synthesize three-dimensional data that represents the object in three dimensions; an unmanned moving body for marker that displays a marker on the unmanned moving body; Equipped with The unmanned mobile body control device generates an observation plan including a movement plan for the marker-use unmanned mobile body so that the marker displayed by the marker-use unmanned mobile body is positioned at a position that becomes a characteristic point in the imaging data obtained by the unmanned mobile body imaging the object. The marker-use unmanned mobile body moves and displays the marker so that the marker is positioned at a position that becomes a characteristic point in accordance with the movement plan for the marker-use unmanned mobile body. The unmanned mobile body further calculates imaging conditions using imaging data in which the marker is imaged together with the object. It is characterized by: [Effects of the Invention]

[0007] The three-dimensional remote sensing system of the present disclosure has the advantage of being able to synthesize three-dimensional data of an object using imaging data captured by an unmanned moving body. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an image of a three-dimensional remote sensing system according to the first embodiment. [Figure 2] FIG. 1 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a first embodiment. [Figure 3]FIG. 1 is a diagram showing a configuration example of an unmanned mobile object control device according to a first embodiment; [Figure 4] FIG. 1 is a diagram showing a configuration example of an unmanned moving body according to a first embodiment; [Figure 5] FIG. 1 shows an example of the configuration of a server according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing a configuration example of a relative position / angle acquisition unit included in the server according to the first embodiment; [Figure 7] 1 is a flowchart showing the operation of the three-dimensional remote sensing system according to the first embodiment. [Figure 8] 1 is a flowchart showing the operation of the unmanned mobile object control device according to the first embodiment. [Figure 9] Flowchart showing the operation of the unmanned moving body according to the first embodiment [Figure 10] 1 is a flowchart showing the operation of a server according to the first embodiment; [Figure 11] FIG. 1 is a diagram showing an example of the configuration of a processing circuit that realizes an unmanned mobile object control device according to the first embodiment when the processing circuit is realized by a processor, a memory, and a communication interface. [Figure 12] FIG. 1 is a diagram showing an example of a processing circuit and a communication interface when the processing circuit that realizes the unmanned mobile object control device according to the first embodiment is configured with dedicated hardware. [Figure 13] FIG. 10 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a second embodiment. [Figure 14] FIG. 10 is a diagram showing a configuration example of an unmanned mobile object control device according to a second embodiment; [Figure 15] FIG. 10 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a third embodiment. [Figure 16] FIG. 10 is a diagram showing a configuration example of an unmanned mobile object control device according to a third embodiment; [Figure 17] FIG. 10 is a diagram showing a configuration example of an unmanned moving body according to a fourth embodiment. [Figure 18] FIG. 13 is a diagram showing an example of the configuration of a server according to the fifth embodiment. [Figure 19] FIG. 13 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a sixth embodiment. [Figure 20]FIG. 13 is a diagram showing a configuration example of an unmanned mobile object control device according to a sixth embodiment. [Figure 21] FIG. 13 is a diagram showing a configuration example of an unmanned moving body for marker use according to a sixth embodiment; [Figure 22] FIG. 13 is a diagram showing a configuration example of an unmanned moving body according to a sixth embodiment. [Figure 23] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object group control system according to a sixth embodiment. [Figure 24] FIG. 13 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a seventh embodiment. [Figure 25] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object control device according to a seventh embodiment. [Figure 26] FIG. 13 is a diagram showing a configuration example of an unmanned vehicle for positioning according to a seventh embodiment; [Figure 27] FIG. 13 is a diagram showing a configuration example of an unmanned moving body according to a seventh embodiment. [Figure 28] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object group control system according to a seventh embodiment. [Figure 29] FIG. 13 is a diagram showing a configuration example of a three-dimensional remote sensing system according to an eighth embodiment. [Figure 30] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object control device according to an eighth embodiment. [Figure 31] FIG. 13 is a diagram showing a configuration example of an unmanned moving body according to an eighth embodiment. [Figure 32] FIG. 13 is a diagram showing a configuration example of a communicating unmanned vehicle according to an eighth embodiment. [Figure 33] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object group control system according to an eighth embodiment. [Figure 34] FIG. 13 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a ninth embodiment. [Figure 35] FIG. 20 is a diagram showing an example of the configuration of a server according to the ninth embodiment. [Figure 36] FIG. 13 is a diagram showing a configuration example of an unmanned mobile object control device according to a ninth embodiment. [Figure 37] FIG. 23 is a diagram showing a configuration example of a three-dimensional remote sensing system according to a tenth embodiment. [Figure 38] FIG. 23 shows an example of the configuration of a server according to a tenth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Below, the 3D remote sensing system, unmanned mobile object control device, unmanned mobile object, unmanned mobile object for marker, unmanned mobile object for positioning, unmanned mobile object for communication, server, unmanned mobile object group control system, control circuit, storage medium, 3D remote sensing system operation method, unmanned mobile object control device operation method, unmanned mobile object operation method, and server operation method relating to embodiments of the present disclosure are described in detail based on the drawings.

[0010] Embodiment 1 FIG. 1 is a diagram illustrating an image of a 3D remote sensing system according to the first embodiment and the like. The 3D remote sensing system is a system in which a server 50 synthesizes 3D data of an object 60 using mobile object data including imaging data of the object 60 captured by an unmanned mobile object 40. The unmanned mobile object 40 moves according to a movement plan included in an observation plan generated by an unmanned mobile object control device 10, and captures images of the object 60. Examples of the object 60 include, but are not limited to, collapsed buildings and fallen trees in the event of a disaster. The 3D remote sensing system may include multiple unmanned mobile objects 40 as shown in FIG. 1, or may include only one unmanned mobile object 40. The unmanned mobile object 40 is, for example, a drone equipped with an imaging sensor such as an optical camera.

[0011] The unmanned mobile body control device 10 transmits an observation plan including a movement plan to the unmanned mobile body 40 via the terrestrial network 20 and the wireless base station 30. The unmanned mobile body 40 also transmits mobile body data including imaging data to the server 50 via the wireless base station 30 and the terrestrial network 20. The unmanned mobile body 40 can determine its own position by receiving signals from positioning satellites 70, but if it cannot receive signals from the positioning satellites 70 due to an obstruction 80 such as a building, it can determine its own position by using a marker-use unmanned mobile body 40E that displays a marker, or a positioning unmanned mobile body 40F that is capable of terminal-to-terminal relative positioning with the unmanned mobile body 40 while receiving signals from the positioning satellites 70.

[0012] Note that the image of the 3D remote sensing system shown in FIG. 1 also includes configurations that will be explained in different embodiments from embodiment 2 onwards, and the 3D remote sensing system does not necessarily have to include all of the configurations shown in FIG. 1. Also, in the explanation of each embodiment, the reference numerals actually assigned to each device may differ from the reference numerals assigned to each device in FIG. 1. Hereinafter, each embodiment will be explained using the necessary components of the components shown in FIG. 1. First, in embodiment 1, the basic operation of the 3D remote sensing system will be explained.

[0013] FIG. 2 is a diagram illustrating an example configuration of a three-dimensional remote sensing system 100 according to the first embodiment. The three-dimensional remote sensing system 100 includes an unmanned mobile object control device 10, a wireless base station 30, an unmanned mobile object 40, and a server 50. Two wireless base stations 30 are shown in FIG. 2 to make the data flow easier to understand. As shown in FIG. 1, if the wireless base station 30 on the communication path between the unmanned mobile object control device 10 and the unmanned mobile object 40 is the same as the wireless base station 30 on the communication path between the unmanned mobile object 40 and the server 50, only one wireless base station 30 may be used. The same applies to the following embodiments. Furthermore, although three unmanned mobile objects 40 are shown in FIG. 2, the three-dimensional remote sensing system 100 may include at least one unmanned mobile object 40.

[0014] The unmanned mobile body control device 10 generates an observation plan including a movement plan for the unmanned mobile body 40, and transmits the generated observation plan to the unmanned mobile body 40 via the ground network 20 and wireless base station 30 shown in FIG. 1. The unmanned mobile body control device 10 also transmits the observation plan to the server 50 via the ground network 20 shown in FIG. 1. Note that if the unmanned mobile body control device 10 can transmit the observation plan to the server 50 without passing through the ground network 20, it may also transmit the observation plan to the server 50 without passing through the ground network 20. There is no particular restriction on the method for transmitting the observation plan from the unmanned mobile body control device 10 to the server 50. Furthermore, the ground network 20 may be a wired network, a wireless network, or a terrestrial cellular network as described below.

[0015] 3 is a diagram showing an example configuration of the unmanned mobile body control device 10 according to embodiment 1. The unmanned mobile body control device 10 includes an observation plan formulation unit 11. The observation plan formulation unit 11 includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112, an image sensor image capture plan generation unit 113, and an observation plan generation unit 114.

[0016] The image capture target position information storage unit 111 outputs image capture target position information, which is position information of the target 60 captured by the unmanned mobile body 40. The image capture target position information storage unit 111 acquires the image capture target position information from, for example, a person in charge of the organization that operates the 3D remote sensing system 100 by wired communication, wireless communication, manual input, or other methods, but the acquisition method is not limited to these. The image capture target position information may include not only image capture target position information for one target 60, but also image capture target position information for multiple targets 60.

[0017] The unmanned mobile object movement plan generation unit 112 generates a movement plan for the unmanned mobile object 40 using the image capture target position information acquired from the image capture target position information storage unit 111. The movement plan includes, for example, the movement route, movement time, and attitude of the unmanned mobile object 40. Furthermore, when multiple unmanned mobile objects 40 are included in the three-dimensional remote sensing system 100 as shown in FIG. 2, the unmanned mobile object movement plan generation unit 112 generates a movement plan for each unmanned mobile object 40. The movement plan for each unmanned mobile object 40 is configured so that, for example, the movement route is different for each unmanned mobile object 40, or, if the movement routes of the unmanned mobile objects 40 are at least partially the same, the movement time on the same movement route is different for each unmanned mobile object 40.

[0018] The imaging sensor imaging plan generation unit 113 generates an imaging plan for the imaging sensor 43 mounted on the unmanned mobile body 40 to capture an image of the object 60, based on the imaging target position information acquired from the imaging target position information storage unit 111 and the movement plan generated by the unmanned mobile body movement plan generation unit 112. The imaging plan includes parameters such as the imaging time of the imaging sensor 43, the angle of the imaging sensor 43, and the zoom.

[0019] The observation plan generation unit 114 generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile body movement plan generation unit 112, and the imaging plan acquired from the imaging sensor imaging plan generation unit 113. Note that the observation plan is not limited to the example described above, and may include only a movement plan without including an imaging plan or imaging target position information. Furthermore, the observation plan may include one of the imaging plan and imaging target position information along with the movement plan. In this way, in the unmanned mobile body control device 10, the observation plan generation unit 114 generates an observation plan that includes at least a movement plan for the unmanned mobile body 40.

[0020] 1 and 2, the observation plan generation unit 114 transmits the generated observation plan to the unmanned mobile body 40 via the terrestrial network 20 and wireless base station 30. The observation plan generation unit 114 also transmits the observation plan to the server 50 via the terrestrial network 20. The unmanned mobile body control device 10 may also be equipped with a dedicated communication device for transmitting the observation plan to the unmanned mobile body 40, server 50, etc.

[0021] Returning to the explanation of Figure 2, the wireless base station 30 transmits the observation plan received from the observation plan generation unit 114 of the unmanned mobile object control device 10 via the terrestrial network 20 to the unmanned mobile object 40. When the three-dimensional remote sensing system 100 includes multiple unmanned mobile objects 40 as shown in Figure 2, the wireless base station 30 transmits the observation plan to each unmanned mobile object 40. Communication between the wireless base station 30 and the unmanned mobile object 40 is, for example, via a terrestrial cellular network, but is not limited to this.

[0022] The unmanned mobile body 40 captures an image of the target object 60 using the imaging sensor 43 based on the observation plan received from the wireless base station 30, and generates mobile body data including the image data and the imaging conditions. The unmanned mobile body 40 transmits the generated mobile body data to the server 50 via the wireless base station 30 and the ground network 20.

[0023] FIG. 4 is a diagram illustrating an example of the configuration of an unmanned mobile body 40 according to the first embodiment. The unmanned mobile body 40 includes a wireless terminal 41, a mobile body control unit 42, an imaging sensor 43, and an imaging condition acquisition unit 44. In FIGS. 2 and 4, two wireless terminals 41 are shown in the unmanned mobile body 40 to facilitate understanding of the data flow. As shown in FIG. 1, when the wireless base station 30 on the communication path between the unmanned mobile body control device 10 and the unmanned mobile body 40 is the same as the wireless base station 30 on the communication path between the unmanned mobile body 40 and the server 50, the unmanned mobile body 40 may have only one wireless terminal 41 that performs wireless communication with the wireless base station 30. Alternatively, the unmanned mobile body 40 may have two wireless terminals 41 for different purposes, such as a wireless terminal 41 for receiving and a wireless terminal 41 for transmitting.

[0024] The wireless terminal 41 receives the observation plan from the wireless base station 30 and outputs the received observation plan to the mobile unit control unit 42.

[0025] The mobile body control unit 42 controls the movement of the unmanned mobile body 40 and the capture of images by the imaging sensor 43 based on the observation plan acquired from the wireless terminal 41. Specifically, the mobile body control unit 42 uses the observation plan to generate mobile body control information for controlling the movement of the unmanned mobile body 40 and generates imaging control information for controlling the capture of images of the target object 60 by the imaging sensor 43. The mobile body control unit 42 outputs the generated mobile body control information, together with the imaging condition acquisition unit 44, to a control unit (not shown) for driving a propeller or the like if the unmanned mobile body 40 is a drone. The mobile body control unit 42 also outputs the generated imaging control information, together with the imaging condition acquisition unit 44, to the imaging sensor 43. Note that, for the sake of brevity, FIG. 4 omits the output of imaging control information from the mobile body control unit 42 to the imaging sensor 43. This also applies to subsequent similar figures. Furthermore, the mobile object control unit 42 generates an object position indicating the position of the object 60 from the imaging object position information included in the observation plan, and outputs the generated object position to the wireless terminal 41. Note that if the observation plan does not include an imaging plan or imaging object position information, the mobile object control unit 42 may set the imaging control information and object position described above to fixed values.

[0026] The imaging sensor 43 captures an image of the object 60 based on imaging control information acquired from the mobile object control unit 42. The imaging sensor 43 is, for example, a sensor such as an optical camera, a stereo camera, or a LiDAR (Light Detection and Ranging) sensor, but is not limited to these. The imaging sensor 43 may also include multiple types of sensors. The imaging sensor 43 outputs imaging data, which is the imaging result obtained by capturing an image of the object 60, to the wireless terminal 41.

[0027] The imaging condition acquisition unit 44 acquires the imaging conditions when the imaging sensor 43 captured an image of the object 60, based on the mobile object control information and imaging control information acquired from the mobile object control unit 42. The imaging conditions include, for example, the position of the imaging sensor 43 when it captured the object 60, the angle of the imaging sensor 43 when it captured the object 60, and the time when it captured the object 60, but are not limited to these and may include other information. For example, the imaging condition acquisition unit 44 can acquire the angle of the imaging sensor 43 when it captured the object 60 by combining an angle indicating the attitude of the unmanned mobile object 40 with the angle of the imaging sensor 43 relative to the unmanned mobile object 40 at the time of image capture, which is indicated by the imaging control information. The imaging condition acquisition unit 44 outputs the acquired imaging conditions of the imaging sensor 43, i.e., information about the position, angle, and time when the imaging sensor 43 captured the object 60, to the wireless terminal 41.

[0028] The wireless terminal 41 acquires the object position indicating the position of the object 60 from the mobile object control unit 42, acquires imaging data, which is the imaging result of the imaging sensor 43, from the imaging sensor 43, and acquires the imaging conditions from the imaging condition acquisition unit 44. The wireless terminal 41 generates mobile object data including the object position, imaging data, and imaging conditions, and transmits the data to the server 50 via the wireless base station 30 and the terrestrial network 20.

[0029] Returning to the explanation of Fig. 2, the wireless base station 30 transmits the mobile body data received from the wireless terminal 41 of the unmanned mobile body 40 to the server 50 via the terrestrial network 20. When the wireless base station 30 receives mobile body data from the wireless terminals 41 of multiple unmanned mobile bodies 40, it transmits all of the mobile body data to the server 50 via the terrestrial network 20.

[0030] The server 50 receives the mobile body data from the unmanned mobile body 40 via the wireless base station 30 and the ground network 20, and receives the observation plan from the unmanned mobile body control device 10 via the ground network 20. The server 50 uses the mobile body data and the observation plan to synthesize three-dimensional data that represents the target object 60 in three dimensions.

[0031] 5 is a diagram showing an example of the configuration of the server 50 according to embodiment 1. The server 50 includes a relative position / angle acquisition unit 51 and a three-dimensional data synthesis unit 52.

[0032] The relative position and angle acquisition unit 51 receives an observation plan including at least a movement plan for the unmanned mobile body 40 from the unmanned mobile body control device 10 via the ground network 20. The relative position and angle acquisition unit 51 also receives from the unmanned mobile body 40 via the wireless base station 30 and the ground network 20, imaging data of the target object 60 captured by the unmanned mobile body 40 using the imaging sensor 43 based on the observation plan, an object position indicating the position of the target object 60, and mobile body data including imaging conditions of the imaging sensor 43 on the unmanned mobile body 40. The server 50 may be equipped with a dedicated communication device for receiving the observation plan, mobile body data, etc. Based on the observation plan and the mobile body data, the relative position and angle acquisition unit 51 outputs the imaging data, the target object position, the relative position of the imaging sensor 43 with respect to the target object 60, the relative angle of the imaging sensor 43 with respect to the target object 60, and the time when the imaging sensor 43 captured the target object 60.

[0033] The three-dimensional data synthesis unit 52 synthesizes three-dimensional data that three-dimensionally represents the object 60, using the imaging data, object position, relative position, relative angle, and time acquired from the relative position / relative angle acquisition unit 51. The three-dimensional data synthesis unit 52 can synthesize the three-dimensional data using, for example, a Neural Radiance Fields (NeRF) method, but is not limited to this and other methods may also be used.

[0034] 6 is a diagram showing an example of the configuration of the relative position / angle acquisition unit 51 included in the server 50 according to Embodiment 1. The relative position / angle acquisition unit 51 includes a data division unit 511 and a relative position / angle calculation unit 512.

[0035] The data division unit 511 receives mobile body data from each unmanned mobile body 40 via the wireless base station 30 and the ground network 20. The data division unit 511 also receives an observation plan from the unmanned mobile body control device 10 via the ground network 20. The data division unit 511 divides the mobile body data into imaging data, object position, the position of the imaging sensor 43 when it imaged the object 60, the angle of the imaging sensor 43 when it imaged the object 60, and the time when the imaging sensor 43 imaged the object 60, and outputs the divided data. If all of the data included in the received mobile body data can be used, the data division unit 511 does not need to use the observation plan. If data such as the position of the imaging sensor 43 equipped on the unmanned mobile body 40 and the angle of the imaging sensor 43 equipped on the unmanned mobile body 40 is not included in the mobile body data or is unavailable, the data division unit 511 may use the observation plan to estimate the position of the imaging sensor 43 equipped on the unmanned mobile body 40 and the angle of the imaging sensor 43 equipped on the unmanned mobile body 40. Furthermore, if data on the target object position is not included in the mobile body data or is unavailable, the data division unit 511 may estimate the target object position from the imaging data. In this way, the data division unit 511 uses the observation plan as supplementary information.

[0036] The relative position and relative angle calculation unit 512 acquires from the data division unit 511 the object position, the position of the imaging sensor 43 provided in the unmanned mobile body 40, the angle of the imaging sensor 43 provided in the unmanned mobile body 40, and the time when the imaging sensor 43 provided in the unmanned mobile body 40 captured an image of the object 60. Using this acquired data, the relative position and relative angle calculation unit 512 calculates the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60. The relative position and relative angle calculation unit 512 outputs the calculated relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60, as well as the time when the imaging sensor 43 captured an image of the object 60.

[0037] In this way, if the server 50 can obtain the necessary data from the mobile body data, it is possible to synthesize three-dimensional data that represents the target object 60 in three dimensions using only the mobile body data without using an observation plan.

[0038] 7 is a flowchart showing the operation of the three-dimensional remote sensing system 100 according to the first embodiment. In the three-dimensional remote sensing system 100, the unmanned mobile object control device 10 generates an observation plan (step S11) and transmits the observation plan to the unmanned mobile object 40 and the server 50. The unmanned mobile object 40 captures an image of the target object 60 using the imaging sensor 43 based on the observation plan (step S12). The unmanned mobile object 40 generates mobile object data including the captured image data (step S13) and transmits the mobile object data to the server 50. The server 50 uses the mobile object data and the observation plan to synthesize three-dimensional data that represents the target object 60 in three dimensions (step S14).

[0039] 8 is a flowchart showing the operation of the unmanned mobile body control device 10 according to the first embodiment. In the unmanned mobile body control device 10, the unmanned mobile body movement plan generation unit 112 generates a movement plan for the unmanned mobile body 40 using the image capture target position information (step S21). The image capture sensor image capture plan generation unit 113 generates an image capture plan for the image capture sensor 43 mounted on the unmanned mobile body 40 to capture an image of the target 60 based on the image capture target position information and the movement plan (step S22). The observation plan generation unit 114 generates an observation plan including the image capture target position information, the movement plan, and the image capture plan (step S23).

[0040] 9 is a flowchart showing the operation of the unmanned mobile body 40 according to the first embodiment. In the unmanned mobile body 40, the wireless terminal 41 receives an observation plan including at least a movement plan for the unmanned mobile body 40 (step S31). The mobile body control unit 42 generates mobile body control information and imaging control information using the observation plan (step S32). The imaging sensor 43 captures an image of the object 60 based on the imaging control information (step S33). The imaging condition acquisition unit 44 acquires the imaging conditions used when the imaging sensor 43 captured an image of the object 60 based on the mobile body control information and the imaging control information (step S34). The wireless terminal 41 transmits mobile body data including the object position indicating the position of the object 60, imaging data which is the imaging result of the imaging sensor 43, and the imaging conditions (step S35).

[0041] 10 is a flowchart showing the operation of the server 50 according to embodiment 1. In the server 50, the relative position / angle acquisition unit 51 outputs the imaging data, the object position, the relative position of the imaging sensor 43 with respect to the object 60, the relative angle of the imaging sensor 43 with respect to the object 60, and the time when the imaging sensor 43 captured the image of the object 60, based on the imaging data, the object position, and the imaging conditions (step S41). The three-dimensional data synthesis unit 52 synthesizes three-dimensional data that represents the object 60 in three dimensions, using the imaging data, the object position, the relative position, the relative angle, and the time (step S42).

[0042] Next, the hardware configuration of each device in the 3D remote sensing system 100 will be described. In the unmanned mobile object control device 10, the function of transmitting an observation plan of the observation plan generation unit 114 included in the observation plan formulation unit 11 is realized by a communication interface. In the unmanned mobile object control device 10, the functions of the image capture target position information storage unit 111, the unmanned mobile object movement plan generation unit 112, the imaging sensor image capture plan generation unit 113, and the observation plan generation unit 114 other than the function of transmitting an observation plan are realized by processing circuits. The processing circuit may be a processor and memory that executes a program stored in memory, or may be dedicated hardware. The processing circuit is also called a control circuit.

[0043] FIG. 11 is a diagram illustrating a configuration example of a processing circuit 90 that implements the unmanned mobile object control device 10 according to the first embodiment when the processing circuit is implemented by a processor 91, a memory 92, and a communication interface 93. The processing circuit 90 illustrated in FIG. 11 is a control circuit and includes a processor 91, a memory 92, and a communication interface 93. When the processing circuit 90 is configured with the processor 91, the memory 92, and the communication interface 93, each function of the processing circuit 90 is implemented by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 92. In the processing circuit 90, each function is implemented by the processor 91 reading and executing the program stored in the memory 92 and transmitting it via the communication interface 93. That is, the processing circuit 90 includes the memory 92 for storing a program that results in the processing of the unmanned mobile object control device 10 being executed. This program can also be said to be a program that causes the unmanned mobile object control device 10 to execute each function implemented by the processing circuit 90. This program may be provided by a storage medium on which the program is stored, or by other means such as a communication medium.

[0044] The above program can also be said to be a program that causes the unmanned mobile body control device 10 to execute a first step in which the unmanned mobile body movement plan generation unit 112 generates a movement plan for the unmanned mobile body 40 using imaged object position information, which is the position information of the object 60 imaged by the unmanned mobile body 40, and a second step in which the observation plan generation unit 114 generates an observation plan that includes at least a movement plan for the unmanned mobile body 40.

[0045] Here, the processor 91 is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor). The memory 92 is, for example, a nonvolatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc). The communication interface 93 is, for example, an interface capable of communication via a LAN (Local Area Network), a USB (Universal Serial Bus), a wireless LAN, mobile communications such as 4G (Fourth Generation) or 5G (Fifth Generation), Bluetooth (registered trademark), or the like.

[0046] FIG. 12 is a diagram illustrating an example of a processing circuit 94 and a communication interface 93 when the processing circuit that realizes the unmanned mobile object control device 10 according to the first embodiment is configured with dedicated hardware. The processing circuit 94 illustrated in FIG. 12 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit may be partially realized with dedicated hardware and partially realized with software or firmware. In this way, the processing circuit can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination thereof.

[0047] The hardware configuration of the unmanned mobile object control device 10 has been described, but the hardware configuration of the unmanned mobile object 40 and the server 50 is also the same.

[0048] In the unmanned mobile object 40, the wireless terminal 41 is realized by a communication interface. The imaging sensor 43 is realized by a sensor such as an optical camera. The mobile object control unit 42 and the imaging condition acquisition unit 44 are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in memory, or may be dedicated hardware. The processing circuit is also called a control circuit.

[0049] In addition, in the server 50, the function of the relative position and angle acquisition unit 51 that receives moving body data and the like is realized by a communication interface. In the server 50, the three-dimensional data synthesis unit 52 and the functions of the relative position and angle acquisition unit 51 other than the function of receiving moving body data and the like are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in memory, or may be dedicated hardware. The processing circuit is also called a control circuit.

[0050] As described above, according to this embodiment, in the 3D remote sensing system 100, the unmanned mobile body control device 10 generates an observation plan including a movement plan for the unmanned mobile body 40. The unmanned mobile body 40 captures images of the object 60 using the imaging sensor 43 based on the observation plan, generates mobile body data including the imaging data, the object position, imaging conditions, etc., and transmits this to the server 50. The server 50 uses the mobile body data and the observation plan to acquire data such as the relative position of the imaging sensor 43 with respect to the object 60, the relative angle of the imaging sensor 43 with respect to the object 60, and the time when the imaging sensor 43 captured the object 60, and synthesizes 3D data that represents the object 60 in three dimensions. This allows the 3D remote sensing system 100 to synthesize 3D data of the object 60 using the imaging data captured by the unmanned mobile body 40. The three-dimensional remote sensing system 100 can synthesize three-dimensional data that represents the object 60 in three dimensions, and is therefore applicable not only to inspection and surveying applications such as those described in Patent Document 1, but also to applications such as remotely grasping the condition of objects 60 such as collapsed buildings and fallen trees when a disaster occurs.

[0051] Embodiment 2 In the first embodiment, the number of unmanned mobile bodies 40, the timing of capturing images of the unmanned mobile bodies 40, etc., were not particularly limited in the three-dimensional remote sensing system 100. In the second embodiment, a case will be described in which one unmanned mobile body 40 captures images of an object 60 multiple times in a three-dimensional remote sensing system. For the sake of simplicity, the second embodiment will be described mainly with respect to the differences from the first embodiment. The same applies to the following embodiments.

[0052] 13 is a diagram showing an example of the configuration of a three-dimensional remote sensing system 100a according to embodiment 2. The three-dimensional remote sensing system 100a is a system that acquires moving body data in a time-sharing manner while moving an unmanned moving body 40. The three-dimensional remote sensing system 100a includes an unmanned moving body control device 10a, a wireless base station 30, an unmanned moving body 40, and a server 50.

[0053] The unmanned mobile object control device 10a generates an observation plan for the unmanned mobile object 40 to capture images of the target object 60 at different times.

[0054] 14 is a diagram showing an example of the configuration of an unmanned mobile body control device 10a according to embodiment 2. The unmanned mobile body control device 10a includes an observation plan formulation unit 11a. The observation plan formulation unit 11a includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112a, an image sensor image capture plan generation unit 113a, and an observation plan generation unit 114.

[0055] The unmanned mobile body movement plan generation unit 112a generates a movement plan for the unmanned mobile body 40 to capture images of the object 60 at different times, using the image capture object position information acquired from the image capture object position information storage unit 111. The movement plan includes, for example, the movement route, movement time, and attitude of the unmanned mobile body 40.

[0056] The imaging sensor imaging plan generation unit 113a generates an imaging plan for the imaging sensor 43 mounted on the unmanned mobile body 40 to capture images of the object 60 multiple times at different times, using the imaging target position information acquired from the imaging target position information storage unit 111 and the movement plan generated by the unmanned mobile body movement plan generation unit 112a. The imaging plan includes parameters such as the imaging time of the imaging sensor 43, the angle of the imaging sensor 43, and the zoom. In the example of Fig. 13, there are three imaging times, t = 0, t = 1, and t = 2, but the number of imaging times may be two or four or more.

[0057] The observation plan generation unit 114 generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile body movement plan generation unit 112a, and the imaging plan acquired from the imaging sensor imaging plan generation unit 113a. Note that the observation plan is not limited to the example described above, and may include a movement plan and an imaging plan without including imaging target position information. In this way, in the unmanned mobile body control device 10a, the observation plan generation unit 114 generates an observation plan that includes at least a movement plan and an imaging plan for the unmanned mobile body 40.

[0058] Returning to the explanation of FIG. 13 , the unmanned mobile body 40 captures images of the object 60 at different times based on an observation plan and generates mobile body data for each image capture time. In the example of FIG. 13 , the unmanned mobile body 40 generates mobile body data for times t=0, t=1, and t=2. The server 50 uses the mobile body data for each image capture time to synthesize three-dimensional data that represents the object 60 in three dimensions. For example, when the object 60 is stationary, the server 50 can synthesize three-dimensional data that represents one object 60 in three dimensions using mobile body data captured at different times, compared to synthesizing three-dimensional data that represents the object 60 in three dimensions using only mobile body data captured at a single time. In other words, since the server 50 can use a large amount of mobile body data, it can synthesize highly accurate three-dimensional data.

[0059] It should be noted that the target object 60 in the second embodiment is not limited to a stationary object 60. The second embodiment is also applicable to cases where the movement of the object 60 is slow and the movement of the object 60 can be ignored relative to the image capturing interval of the unmanned mobile body 40. Furthermore, the second embodiment is also applicable to cases where the object 60 moves at a constant cycle, because it is possible to capture changes in the object 60 by adjusting the image capturing interval of the unmanned mobile body 40 to the constant cycle.

[0060] As described above, according to this embodiment, in the 3D remote sensing system 100a, the unmanned mobile body control device 10a generates an observation plan for the unmanned mobile body 40 to capture images of the object 60 at different times. The unmanned mobile body 40 captures images of the object 60 using the imaging sensor 43 based on the observation plan and generates mobile body data for each image capture time. The server 50 then uses the mobile body data for each image capture time to synthesize 3D data that represents the object 60 in three dimensions. This allows the 3D remote sensing system 100a to synthesize 3D data with higher accuracy than in the first embodiment. Even when the 3D remote sensing system 100a includes only one unmanned mobile body 40, the unmanned mobile body 40 can capture images of the object 60 multiple times, thereby synthesizing 3D data of the object 60.

[0061] Embodiment 3 In the first embodiment, there are no particular limitations on the number of unmanned mobile bodies 40 or the timing of capturing images of the unmanned mobile bodies 40 in the three-dimensional remote sensing system 100. In the third embodiment, a case will be described in which a plurality of unmanned mobile bodies 40 simultaneously capture images of an object 60 in a three-dimensional remote sensing system.

[0062] 15 is a diagram showing an example of the configuration of a three-dimensional remote sensing system 100b according to the third embodiment. The three-dimensional remote sensing system 100b is a system that simultaneously acquires data on a plurality of unmanned mobile bodies 40-1 to 40-3 while moving the unmanned mobile bodies 40-1 to 40-3. The three-dimensional remote sensing system 100b includes an unmanned mobile body control device 10b, a wireless base station 30, unmanned mobile bodies 40-1 to 40-3, and a server 50. The configuration of the unmanned mobile bodies 40-1 to 40-3 is the same as the configuration of the unmanned mobile body 40 according to the first and second embodiments.

[0063] The unmanned mobile object control device 10b generates an observation plan for the plurality of unmanned mobile objects 40-1 to 40-3 to capture images of the target object 60.

[0064] 16 is a diagram showing an example configuration of an unmanned mobile body control device 10b according to embodiment 3. The unmanned mobile body control device 10b includes an observation plan formulation unit 11b. The observation plan formulation unit 11b includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112b, an image sensor image capture plan generation unit 113b, and an observation plan generation unit 114.

[0065] The unmanned mobile body movement plan generation unit 112b generates a movement plan for the multiple unmanned mobile bodies 40-1 to 40-3 to capture images of the object 60, based on the image capture object position information acquired from the image capture object position information storage unit 111. The movement plan includes, for example, the movement routes, movement times, and attitudes of the multiple unmanned mobile bodies 40-1 to 40-3.

[0066] The imaging sensor imaging plan generation unit 113b uses the imaging target position information acquired from the imaging target position information storage unit 111 and the movement plan generated by the unmanned mobile body movement plan generation unit 112b to generate an imaging plan for each imaging sensor 43 mounted on the unmanned mobile bodies 40-1 to 40-3 to capture an image of the target 60. The imaging plan includes parameters for each imaging sensor 43 mounted on the plurality of unmanned mobile bodies 40-1 to 40-3, such as the imaging time of the imaging sensor 43, the angle of the imaging sensor 43, and the zoom.

[0067] The observation plan generation unit 114 generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile body movement plan generation unit 112b, and the imaging plan acquired from the imaging sensor imaging plan generation unit 113b. Note that the observation plan is not limited to the example described above, and may include a movement plan and an imaging plan, but not include imaging target position information. In this way, in the unmanned mobile body control device 10b, the observation plan generation unit 114 generates an observation plan that includes movement plans and imaging plans for at least the unmanned mobile bodies 40-1 to 40-3.

[0068] Returning to the explanation of Figure 15, the multiple unmanned mobile bodies 40-1 to 40-3 capture images of the target object 60 based on an observation plan and generate mobile body data. The multiple unmanned mobile bodies 40-1 to 40-3 capture images at the same time, for example, to generate mobile body data. The server 50 uses the mobile body data received from the multiple unmanned mobile bodies 40-1 to 40-3 to synthesize three-dimensional data that represents the target object 60 in three dimensions. For example, by having the multiple unmanned mobile bodies 40-1 to 40-3 capture images simultaneously, the server 50 can acquire the mobile body data for synthesizing the three-dimensional data in a short time.

[0069] In the second embodiment, it was assumed that the target would be a stationary object 60 or an object 60 that moves slowly, but in the third embodiment, it is assumed that multiple unmanned mobile bodies 40-1 to 40-3 will simultaneously capture images of the object 60, so that it is also possible to target a moving object 60, for example, a moving object 60.

[0070] As described above, according to this embodiment, in the three-dimensional remote sensing system 100b, the unmanned mobile object control device 10b generates an observation plan for the multiple unmanned mobile objects 40-1 to 40-3 to capture images of the object 60. The multiple unmanned mobile objects 40-1 to 40-3 capture images of the object 60 using the imaging sensor 43 based on the observation plan, and generate mobile object data. The server 50 then uses the mobile object data received from the multiple unmanned mobile objects 40-1 to 40-3 to synthesize three-dimensional data that represents the object 60 in three dimensions. This allows the three-dimensional data to be synthesized in a shorter time than in the second embodiment.

[0071] Although the third embodiment has been described using an example in which it is applied to the first embodiment, it can also be applied to the second embodiment. That is, the three-dimensional remote sensing system 100b can capture images of the object 60 from each of the plurality of unmanned mobile bodies 40-1 to 40-3 at a plurality of specified times, and can synthesize three-dimensional data representing the object 60 in three dimensions using the acquired mobile body data. The mobile body data transmitted from the unmanned mobile bodies 40-1 to 40-3 includes time data as an imaging condition. Therefore, when the object 60 is stationary, the server 50 can synthesize one piece of three-dimensional data representing the object 60 in three dimensions using mobile body data from different times. Furthermore, when the object 60 is moving, the server 50 can synthesize three-dimensional data for each time representing the object 60 in three dimensions, which shows the movement of the object 60 over time, using the mobile body data for each time, as long as the unmanned mobile bodies 40-1 to 40-3 capture images multiple times at the same time.

[0072] Embodiment 4 In the fourth embodiment, a specific configuration of an image capturing condition acquisition unit provided in an unmanned moving body will be described.

[0073] 17 is a diagram showing a configuration example of an unmanned mobile body 40c according to embodiment 4. The unmanned mobile body 40c includes a wireless terminal 41, a mobile body control unit 42, an imaging sensor 43, and an imaging condition acquisition unit 44c. The imaging condition acquisition unit 44c includes a GNSS (Global Navigation Satellite System) receiver 441, an inertial sensor 442, an oscillator 443, and a position, angle, and time calculation unit 444.

[0074] The GNSS receiver 441 receives signals from the positioning satellites 70 shown in Fig. 1 and outputs GNSS data including position data based on latitude and longitude and time data. Although Fig. 17 shows only one GNSS receiver 441, the unmanned mobile body 40c may have multiple GNSS receivers 441 and obtain GNSS data from different positions within the unmanned mobile body 40c, thereby making it possible to obtain angle data in addition to position data. In the following description, the GNSS may be referred to as a satellite positioning system.

[0075] The inertial sensor 442 detects three-dimensional inertial motion and outputs movement history data based on acceleration and angular velocity. The inertial sensor 442 is, for example, an IMU (Inertial Measurement Unit). The three-dimensional inertial motion refers to translational motion and rotational motion in three orthogonal axial directions. The movement history data includes tilt, displacement, speed, acceleration, etc.

[0076] The oscillator 443 outputs a clock that serves as a time reference.

[0077] Based on the GNSS data, movement history data, clock, mobile object control information, and imaging control information, the position, angle, and time calculation unit 444 calculates, as imaging conditions, the position of the imaging sensor 43 when it captured an image of the object 60, the angle of the imaging sensor 43 when it captured an image of the object 60, and the time when the imaging sensor 43 captured an image of the object 60. Note that if part of the data is missing from the GNSS data, movement history data, clock, mobile object control information, and imaging control information, the position, angle, and time calculation unit 444 may supplement the data by processing such as interpolation.

[0078] As described above, according to this embodiment, the imaging condition acquisition unit 44c of the unmanned mobile body 40c can obtain information such as the position, angle, and time when the imaging sensor 43 mounted on the unmanned mobile body 40c images the target object 60 under conditions in which signals from the positioning satellite 70 used in GNSS can be received.

[0079] Embodiment 5. In the fourth embodiment, the unmanned mobile body 40c receives signals from the positioning satellites 70 using the GNSS receiver 441 to acquire GNSS data, but if it is unable to receive signals from the positioning satellites 70, it is unable to acquire accurate GNSS data. Even in such a case, the unmanned mobile body 40c can use the movement history data from the inertial sensor 442, but errors may accumulate over time. In the fifth embodiment, a case will be described in which, when the accuracy of information such as position and angle included in the mobile body data from the unmanned mobile body 40c is low, the server estimates the relative position and relative angle of the image sensor 43 with respect to the target object 60 by estimating the viewpoint angle from multiple pieces of image data.

[0080] 18 is a diagram illustrating a configuration example of a server 50d according to embodiment 5. The server 50d includes a relative position / angle acquisition unit 51d and a three-dimensional data synthesis unit 52. The relative position / angle acquisition unit 51d includes a data division unit 511, a relative position / angle calculation unit 512d, and a viewpoint angle estimation unit 513. As described above, the imaging conditions included in the mobile body data transmitted from the unmanned mobile body 40c include the position of the imaging sensor 43 when it captured an image of the object 60, the angle of the imaging sensor 43 when it captured an image of the object 60, and the time when the imaging sensor 43 captured an image of the object 60.

[0081] As in embodiment 1, the data division unit 511 divides the moving body data into imaging data, object position, the position of the imaging sensor 43 when the imaging sensor 43 imaged the object 60, the angle of the imaging sensor 43 when the imaging sensor 43 imaged the object 60, and the time when the imaging sensor 43 imaged the object 60, and outputs the divided data.

[0082] The viewpoint angle estimation unit 513 uses a plurality of pieces of imaging data to estimate, for each piece of imaging data, the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60. The viewpoint angle estimation unit 513 outputs the estimated relative position and relative angle as an estimated position and estimated angle. The viewpoint angle estimation unit 513 can perform the above estimation by using, for example, a Structure from Motion (SfM) technique, but is not limited to this and other techniques may also be used.

[0083] The relative position / angle calculation unit 512d calculates the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60, using the object position, position, angle, and time acquired from the data division unit 511 and the estimated position and estimated angle acquired from the viewpoint angle estimation unit 513. The relative position / angle calculation unit 512d outputs the calculated relative position, relative angle, and time to the three-dimensional data synthesis unit 52. Even if the accuracy of the position and angle acquired from the data division unit 511 is low, the relative position / angle calculation unit 512d can calculate the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60 with higher accuracy than the relative position / angle calculation unit 512 of the first embodiment by using the estimated position and estimated angle acquired from the viewpoint angle estimation unit 513.

[0084] As described above, according to this embodiment, in the relative position / angle acquisition unit 51d of the server 50d, the viewpoint angle estimation unit 513 estimates the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60, and the relative position / angle calculation unit 512d calculates the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60 using the estimated position and estimated angle from the viewpoint angle estimation unit 513. As a result, even if the accuracy of the position and angle included in the mobile body data from the unmanned mobile body 40c is low, the server 50d can accurately calculate the relative position of the imaging sensor 43 with respect to the object 60 and the relative angle of the imaging sensor 43 with respect to the object 60, and can generate three-dimensional data of the object 60.

[0085] In addition, in the case where the time data divided by the data dividing unit 511 cannot be read or is missing, and the imaging data includes information on the time when the imaging sensor 43 captured the image of the object 60, the relative position / angle acquiring unit 51d may have the viewpoint angle estimating unit 513 acquire the time information from the imaging data and output it to the relative position / angle calculating unit 512d.

[0086] Embodiment 6 In the fifth embodiment, when accurate GNSS data cannot be acquired in the fourth embodiment, the server 50d estimates the relative position and relative angle of the imaging sensor 43 with respect to the target object 60 by estimating the viewpoint angle using the imaging data. In the sixth embodiment, a case will be described in which an unmanned moving body for markers that displays markers is used as a three-dimensional remote sensing system when a sufficient improvement effect cannot be obtained by the method in the fifth embodiment.

[0087] 19 is a diagram showing a configuration example of a three-dimensional remote sensing system 100e according to embodiment 6. The three-dimensional remote sensing system 100e includes an unmanned mobile object control device 10e, a wireless base station 30, an unmanned mobile object 40e, a marker-use unmanned mobile object 40E, and a server 50.

[0088] The unmanned mobile body control device 10e generates an observation plan including a movement plan for the marker-use unmanned mobile body 40E so that the marker displayed by the marker-use unmanned mobile body 40E is positioned at a position that is a characteristic point in the imaging data obtained when the unmanned mobile body 40e images the target object 60.

[0089] 20 is a diagram showing an example configuration of an unmanned mobile body control device 10e according to embodiment 6. The unmanned mobile body control device 10e includes an observation plan formulation unit 11e. The observation plan formulation unit 11e includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112, an image sensor image capture plan generation unit 113, a marker-use unmanned mobile body movement plan generation unit 115, and an observation plan generation unit 114e.

[0090] The marker unmanned mobile body movement plan generation unit 115 generates a movement plan for the marker unmanned mobile body 40E so that the marker displayed by the marker unmanned mobile body 40E is positioned at a position that is a characteristic point in the imaging data obtained when the unmanned mobile body 40e images the target object 60.

[0091] The observation plan generation unit 114e generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile body movement plan generation unit 112, the imaging plan acquired from the imaging sensor imaging plan generation unit 113, and the movement plan for the marker-use unmanned mobile body 40E acquired from the marker-use unmanned mobile body movement plan generation unit 115. Note that the observation plan is not limited to the example described above, and may be configured not to include at least one of the imaging plan and the imaging target position information. In this way, the observation plan generation unit 114e generates an observation plan that includes at least the movement plan for the unmanned mobile body 40e acquired from the unmanned mobile body movement plan generation unit 112, and the movement plan for the marker-use unmanned mobile body 40E acquired from the marker-use unmanned mobile body movement plan generation unit 115.

[0092] Returning to the explanation of Figure 19, the marker-use unmanned moving body 40E displays a marker on the unmanned moving body 40e. The marker-use unmanned moving body 40E moves in accordance with the movement plan of the marker-use unmanned moving body 40E so that the marker is placed at a position that becomes a feature point, and displays the marker.

[0093] 21 is a diagram showing a configuration example of a marker-use unmanned moving body 40E according to embodiment 6. The marker-use unmanned moving body 40E includes a wireless terminal 41, a marker-use moving body control unit 42E, a marker display unit 45, and a position, angle, and time acquisition unit 46.

[0094] The wireless terminal 41, like the wireless terminal 41 provided in the unmanned mobile body 40, receives the observation plan from the unmanned mobile body control device 10e via the ground network 20 and the wireless base station 30.

[0095] The marker-use mobile body control unit 42E controls the operation of the marker-use unmanned mobile body 40E based on a movement plan for the marker-use unmanned mobile body 40E, which displays a marker on the unmanned mobile body 40e that images the target object 60, and which is generated so that the marker displayed by the marker-use unmanned mobile body 40E is positioned at a position that is a characteristic point in the imaging data obtained when the unmanned mobile body 40e images the target object 60.

[0096] The marker display unit 45 displays a marker. The marker displayed by the marker display unit 45 may be a marker with fixed display content, or may be an electronic marker containing data such as location and time displayed on an electronic display device. The marker displayed by the marker display unit 45 may be an AR (Augmented Reality) marker.

[0097] The position, angle, and time acquisition unit 46 acquires the position, angle, current time, and the like of the marker unmanned moving body 40E. The position, angle, and time acquisition unit 46 has a configuration similar to that of the imaging condition acquisition unit 44c provided in the unmanned moving body 40c described in embodiment 4, for example. For example, if the marker unmanned moving body 40E does not move according to the movement plan, the position, angle, and time acquisition unit 46 may display the acquired data of the marker unmanned moving body 40E, such as the position, angle, and current time, on the marker display unit 45, or may transmit the data from the wireless terminal 41 to the server 50 via the wireless base station 30 and the ground network 20. In addition, when the unmanned mobile body for marker 40E moves according to the movement plan and the marker is displayed on the marker display unit 45, the position, angle, time acquisition unit 46 does not need to display the acquired data such as the position, angle, and current time of the unmanned mobile body for marker 40E on the marker display unit 45, nor does it need to transmit the data from the wireless terminal 41 to the server 50 via the wireless base station 30 and the terrestrial network 20.

[0098] Returning to the description of Fig. 19, the unmanned moving body 40e further calculates the imaging conditions using imaging data obtained by imaging the target object 60 and the marker displayed by the marker unmanned moving body 40E.

[0099] 22 is a diagram showing a configuration example of an unmanned mobile body 40e according to embodiment 6. The unmanned mobile body 40e includes a wireless terminal 41, a mobile body control unit 42, an imaging sensor 43, and an imaging condition acquisition unit 44e. The imaging condition acquisition unit 44e includes a GNSS receiver 441, an inertial sensor 442, an oscillator 443, a viewpoint angle estimation unit 445, and a position angle and time calculation unit 444e.

[0100] The viewpoint angle estimation unit 445 estimates the relative position of the imaging sensor 43 with respect to the marker and the relative angle of the imaging sensor 43 with respect to the marker using imaging data obtained by capturing an image of a marker of the marker-use unmanned mobile body 40E, which displays a marker on the unmanned mobile body 40e along with the target object 60, and outputs the estimated relative position and relative angle as an estimated position and estimated angle. The viewpoint angle estimation unit 445 only needs to use an object that serves as a feature point in the vicinity of the target object 60. Therefore, in addition to a method of using the marker displayed on the marker-use unmanned mobile body 40E as a feature point, the viewpoint angle estimation unit 445 may use an existing building or the like as a feature point, or a marker that was installed in advance instead of a marker displayed on the marker-use unmanned mobile body 40E may be used as a feature point. The viewpoint angle estimation unit 445 can perform the above-mentioned estimation by using, for example, an SfM technique, similar to the viewpoint angle estimation unit 513 of the server 50d described in the fifth embodiment, but is not limited to this and other techniques may be used. The viewpoint angle estimation unit 445 estimates the viewpoint angle using a single piece of imaging data, and can obtain the relative position and angle of the imaging sensor 43 with respect to the feature point.

[0101] The position, angle, and time calculation unit 444e uses the estimated position and estimated angle in addition to the data acquired by the position, angle, and time calculation unit 444 described in the fourth embodiment to calculate, as imaging conditions, the position of the imaging sensor 43 when the imaging sensor 43 captured an image of the object 60, the angle of the imaging sensor 43 when the imaging sensor 43 captured an image of the object 60, and the time when the imaging sensor 43 captured an image of the object 60. The position, angle, and time calculation unit 444e can calculate a position and angle with higher accuracy by calculating the position and angle using more data than the position, angle, and time calculation unit 444 described in the fourth embodiment. The position, angle, and time calculation unit 444e may add the estimated position acquired from the viewpoint angle estimation unit 445 to a position calculated using the same method as the position, angle, and time calculation unit 444 and output the result as a position. The position, angle, and time calculation unit 444e may also add the estimated angle acquired from the viewpoint angle estimation unit 445 to an angle calculated using the same method as the position, angle, and time calculation unit 444 and output the result as an angle.

[0102] In the sixth embodiment, the three-dimensional remote sensing system 100e includes the marker-use unmanned mobile body 40E in addition to the unmanned mobile body 40e, but this is not limiting. For example, the unmanned mobile body 40e may be provided with a marker display unit 45, so that at least one of the multiple unmanned mobile bodies 40e operates as the marker-use unmanned mobile body 40E.

[0103] As described above, according to this embodiment, the unmanned mobile body 40e calculates the imaging conditions using the markers displayed on the marker-use unmanned mobile body 40E as feature points and transmits the mobile body data. As a result, even if the accuracy of the GNSS receiver 441, the inertial sensor 442, etc. is low, the unmanned mobile body 40e can calculate the imaging conditions using the imaging data, thereby obtaining highly accurate data about the position of the imaging sensor 43 when it captured the object 60 and the angle of the imaging sensor 43 when it captured the object 60.

[0104] The three-dimensional remote sensing system 100e can be considered as a system for controlling the movement, i.e., flight, of the unmanned mobile body 40e using the marker-use unmanned mobile body 40E. Figure 23 is a diagram showing an example configuration of an unmanned mobile body group control system 101e according to embodiment 6. The unmanned mobile body group control system 101e includes an unmanned mobile body control device 10e, a wireless base station 30, an unmanned mobile body 40e, and a marker-use unmanned mobile body 40E.

[0105] The unmanned mobile body control device 10e generates a control plan including at least a movement plan for the unmanned mobile body 40e, and a movement plan for the marker-use unmanned mobile body 40E such that the marker displayed by the marker-use unmanned mobile body 40E is positioned at a position that is a characteristic point in the imaging data obtained when the unmanned mobile body 40e images the target object 60.

[0106] The marker-use unmanned moving body 40E moves in accordance with the movement plan of the marker-use unmanned moving body 40E so that the marker is located at a position that becomes a characteristic point, and displays the marker on the unmanned moving body 40e.

[0107] The unmanned mobile body 40e captures an image of the object 60 using the imaging sensor 43 based on the control plan, calculates the imaging conditions using the imaging data in which the marker is captured together with the object 60, and generates mobile body data including the imaging data and the imaging conditions.

[0108] Embodiment 7 In the fifth embodiment, when accurate GNSS data cannot be acquired in the fourth embodiment, the server 50d estimates the relative position and relative angle of the imaging sensor 43 with respect to the object 60 by estimating the viewpoint angle using the imaging data. In the seventh embodiment, a case will be described in which, when a sufficient improvement effect cannot be obtained by the method of the fifth embodiment, an unmanned mobile body for positioning is used to perform terminal-to-terminal relative positioning between two unmanned mobile bodies as a three-dimensional remote sensing system.

[0109] 24 is a diagram showing a configuration example of a three-dimensional remote sensing system 100f according to embodiment 7. The three-dimensional remote sensing system 100f includes an unmanned mobile object control device 10f, a wireless base station 30, an unmanned mobile object 40f, a positioning unmanned mobile object 40F, and a server 50.

[0110] The unmanned mobile body control device 10f generates an observation plan including a movement plan for the positioning unmanned mobile body 40F, which positions the positioning unmanned mobile body 40F at a position where the positioning unmanned mobile body 40F can receive signals from the positioning satellite 70 to determine the position of the positioning unmanned mobile body 40F, and where terminal-to-terminal relative positioning can be performed between the positioning unmanned mobile body 40F and an unmanned mobile body 40f that cannot receive signals from the positioning satellite 70.

[0111] 25 is a diagram showing an example configuration of an unmanned mobile body control device 10f according to embodiment 7. The unmanned mobile body control device 10f includes an observation plan formulation unit 11f. The observation plan formulation unit 11f includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112, an image capture sensor image capture plan generation unit 113, a positioning unmanned mobile body movement plan generation unit 116, and an observation plan generation unit 114f.

[0112] The positioning unmanned mobile body movement plan generation unit 116 generates a movement plan for the positioning unmanned mobile body 40F so that the positioning unmanned mobile body 40F, which performs terminal-to-terminal relative positioning to position the terminal-to-terminal relative position between the unmanned mobile body 40f and the positioning unmanned mobile body 40F, can receive signals from a positioning satellite 70 to position the position of the positioning unmanned mobile body 40F, and can perform terminal-to-terminal relative positioning between the positioning unmanned mobile body 40F and the unmanned mobile body 40f that cannot receive signals from the positioning satellite 70.

[0113] The observation plan generation unit 114f generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile body movement plan generation unit 112, the imaging plan acquired from the imaging sensor imaging plan generation unit 113, and the movement plan for the positioning unmanned mobile body 40F acquired from the positioning unmanned mobile body movement plan generation unit 116. Note that the observation plan is not limited to the example described above, and may be configured not to include at least one of the imaging plan and the imaging target position information. In this way, the observation plan generation unit 114f generates an observation plan that includes at least the movement plan for the unmanned mobile body 40f and the movement plan for the positioning unmanned mobile body 40F.

[0114] Returning to the explanation of Fig. 24, the positioning unmanned mobile body 40F performs terminal-to-terminal relative positioning to determine the terminal-to-terminal relative position between the unmanned mobile body 40f and the positioning unmanned mobile body 40F. The positioning unmanned mobile body 40F moves according to the movement plan of the positioning unmanned mobile body 40F, and performs terminal-to-terminal relative positioning with the unmanned mobile body 40f that cannot receive signals from the positioning satellite 70.

[0115] 26 is a diagram showing a configuration example of a positioning unmanned mobile body 40F according to embodiment 7. The positioning unmanned mobile body 40F includes a wireless terminal 41, a positioning mobile body control unit 42F, an inter-terminal relative positioning unit 47, and a position angle and time acquisition unit 48.

[0116] The wireless terminal 41 receives the observation plan from the unmanned mobile object control device 10 f via the ground network 20 and the wireless base station 30 , similar to the wireless terminal 41 provided in the unmanned mobile object 40 .

[0117] The positioning mobile body control unit 42F controls the operation of the positioning unmanned mobile body 40F and generates mobile body control information for controlling the movement of the positioning unmanned mobile body 40F based on a movement plan for the positioning unmanned mobile body 40F that positions the positioning unmanned mobile body 40F at a position where the positioning unmanned mobile body 40F can receive signals from the positioning satellite 70 to position the position of the positioning unmanned mobile body 40F, and where terminal-to-terminal relative positioning can be performed to position the terminal-to-terminal relative position between the positioning unmanned mobile body 40F and an unmanned mobile body 40f that images the target object 60 and cannot receive signals from the positioning satellite 70.

[0118] The inter-terminal relative positioning unit 47 performs inter-terminal relative positioning to determine the inter-terminal relative position between the positioning unmanned mobile unit 40F and the unmanned mobile unit 40f that cannot receive signals from the positioning satellite 70. The inter-terminal relative positioning unit 47 determines the relative position between the terminals, i.e., between the positioning unmanned mobile unit 40F and the unmanned mobile unit 40f, by performing communication between the terminals using relative positioning signals. The inter-terminal relative positioning can be achieved by a ranging function such as the IEEE (Institute of Electrical and Electronics Engineers) UWB (Ultra Wide Band) standard or the 3GPP (Third Generation Partnership Project) PC5 standard. The inter-terminal relative positioning can be achieved by using inter-terminal relative positioning with multiple antennas to obtain angle data in addition to the relative position.

[0119] The position, angle, and time acquisition unit 48 acquires the position, angle, current time, etc. of the positioning unmanned vehicle 40F. The position, angle, and time acquisition unit 48 includes a GNSS receiver 481, an inertial sensor 482, an oscillator 483, and a position, angle, and time calculation unit 484.

[0120] The GNSS receiver 481 receives signals from the positioning satellites 70 and outputs GNSS data including position data based on latitude and longitude and time data.

[0121] The inertial sensor 482 detects three-dimensional inertial motion and outputs movement history data based on acceleration and angular velocity. The inertial sensor 442 is, for example, an IMU. The three-dimensional inertial motion refers to translational motion and rotational motion in three orthogonal axial directions. The movement history data includes tilt, displacement, velocity, acceleration, etc.

[0122] The oscillator 483 outputs a clock that serves as a time reference.

[0123] The position angle time calculation unit 484 calculates the position of the positioning unmanned mobile body 40F, the angle of the positioning unmanned mobile body 40F, and the time when the position of the positioning unmanned mobile body 40F and the angle of the positioning unmanned mobile body 40F are calculated based on GNSS data, movement history data, clock, mobile body control information, and the relative position between terminals.

[0124] The wireless terminal 41 acquires the object position indicating the position of the object 60 from the positioning mobile object control unit 42F, and acquires the position, angle, and calculation time of the unmanned mobile object for positioning 40F from the position, angle, and time acquisition unit 48. The wireless terminal 41 generates mobile object data including the object position, the position, angle, and calculation time of the unmanned mobile object for positioning 40F, and transmits it to the server 50 via the wireless base station 30 and the ground network 20.

[0125] Returning to the explanation of Fig. 24, the unmanned mobile body 40f that cannot receive signals from the positioning satellite 70 moves according to the movement plan of the unmanned mobile body 40f, performs inter-terminal relative positioning to determine the inter-terminal relative position with the positioning unmanned mobile body 40F, and further calculates the imaging conditions using the inter-terminal relative position.

[0126] 27 is a diagram showing a configuration example of an unmanned mobile body 40f according to Embodiment 7. The unmanned mobile body 40f includes a wireless terminal 41, a mobile body control unit 42, an inter-terminal relative positioning unit 47, an imaging sensor 43, and an imaging condition acquisition unit 44f. The imaging condition acquisition unit 44f includes a GNSS receiver 441, an inertial sensor 442, an oscillator 443, and a position, angle, and time calculation unit 444f.

[0127] The inter-terminal relative positioning unit 47 performs terminal-to-terminal relative positioning to determine the inter-terminal relative position between the unmanned mobile unit 40f and the positioning unmanned mobile unit 40F, which receives signals from positioning satellites 70 and determines the position of the positioning unmanned mobile unit 40F. When the GNSS receiver 441 is unable to receive signals from positioning satellites 70, the inter-terminal relative positioning unit 47 performs terminal-to-terminal relative positioning with the positioning unmanned mobile unit 40F. The operation of the inter-terminal relative positioning unit 47 provided in the unmanned mobile unit 40f is similar to the operation of the inter-terminal relative positioning unit 47 provided in the positioning unmanned mobile unit 40F. Note that the inter-terminal relative positioning unit 47 provided in the unmanned mobile unit 40f does not need to perform inter-terminal relative positioning when the GNSS receiver 441 provided in the unmanned mobile unit 40f is able to receive signals from positioning satellites 70.

[0128] The position, angle, and time calculation unit 444f further uses the inter-terminal relative position to calculate, as imaging conditions, the position of the imaging sensor 43 when the imaging sensor 43 images the object 60, the angle of the imaging sensor 43 when the imaging sensor 43 images the object 60, and the time when the imaging sensor 43 images the object 60. The position, angle, and time calculation unit 444f can calculate a position and angle with higher accuracy by calculating the position angle using more data than the position, angle, and time calculation unit 444 described in embodiment 4. The position, angle, and time calculation unit 444f may add a position obtained from the inter-terminal relative position to a position calculated using the same method as the position, angle, and time calculation unit 444 and output it as a position. The position, angle, and time calculation unit 444f may also add an angle obtained from the inter-terminal relative position to an angle calculated using the same method as the position, angle, and time calculation unit 444 and output it as an angle.

[0129] In the seventh embodiment, the three-dimensional remote sensing system 100f includes the positioning unmanned vehicle 40F in addition to the unmanned vehicle 40f, but this is not limiting. For example, at least one of the multiple unmanned vehicles 40f may operate as the positioning unmanned vehicle 40F.

[0130] As described above, according to the present embodiment, when unmanned mobile body 40f cannot receive signals from positioning satellites 70 and the positioning accuracy of its own position, angle, etc. is considered to be low, unmanned mobile body 40f performs terminal-to-terminal relative positioning with positioning unmanned mobile body 40F, and calculates its own position, angle, etc. using the terminal-to-terminal relative position obtained as a result of the terminal-to-terminal relative positioning. As a result, even when the accuracy of the GNSS data from GNSS receiver 441 is low, unmanned mobile body 40f can obtain highly accurate data about the position of imaging sensor 43 when imaging object 60 and the angle of imaging sensor 43 when imaging object 60 by calculating the imaging conditions using the terminal-to-terminal relative position.

[0131] The three-dimensional remote sensing system 100f can be considered as a system for controlling the movement, i.e., flight, of the unmanned mobile body 40f using the positioning unmanned mobile body 40F. Figure 28 is a diagram showing an example configuration of an unmanned mobile body group control system 101f according to embodiment 7. The unmanned mobile body group control system 101f includes an unmanned mobile body control device 10f, a wireless base station 30, an unmanned mobile body 40f, and a positioning unmanned mobile body 40F.

[0132] The unmanned mobile body control device 10f generates a control plan including at least a movement plan for the unmanned mobile body 40f, and a movement plan for the positioning unmanned mobile body 40F, which performs terminal-to-terminal relative positioning to position the terminal-to-terminal relative position between the unmanned mobile body 40f and the positioning unmanned mobile body 40F, and which positions the positioning unmanned mobile body 40F at a position where terminal-to-terminal relative positioning can be performed between the positioning unmanned mobile body 40F and an unmanned mobile body 40f that cannot receive signals from the positioning satellite 70.

[0133] The unmanned mobile body for positioning 40F moves according to the movement plan of the unmanned mobile body for positioning 40F and performs terminal-to-terminal relative positioning with the unmanned mobile body 40f that cannot receive signals from the positioning satellite 70.

[0134] The unmanned mobile body 40f moves according to its movement plan, and when it cannot receive a signal from the positioning satellite 70, it performs terminal-to-terminal relative positioning with the positioning unmanned mobile body 40F, and calculates the position and angle of the unmanned mobile body 40f using the terminal-to-terminal relative position. Note that the unmanned mobile body group control system 101f can also be applied to unmanned mobile bodies that do not have an imaging sensor 43.

[0135] Embodiment 8 In embodiment 8, we will explain the case where, when there is an unmanned mobile body that cannot communicate wirelessly with the radio base station 30, a communicating unmanned mobile body that can communicate wirelessly with the radio base station 30 relays wireless communication between the radio base station 30 and the unmanned mobile body that cannot communicate wirelessly with the radio base station 30.

[0136] 29 is a diagram showing a configuration example of a three-dimensional remote sensing system 100g according to embodiment 8. The three-dimensional remote sensing system 100g includes an unmanned mobile object control device 10g, a wireless base station 30, an unmanned mobile object 40g, a communication unmanned mobile object 40G, and a server 50.

[0137] The unmanned mobile body control device 10g generates an observation plan including a movement plan for the communication unmanned mobile body 40G, which positions the communication unmanned mobile body 40G in a position where it can communicate with a radio base station 30 on the communication path when the communication unmanned mobile body 40G transmits mobile body data to the server 50, and where terminal-to-terminal communication is possible between the communication unmanned mobile body 40G and an unmanned mobile body 40g that cannot communicate with the radio base station 30.

[0138] 30 is a diagram showing a configuration example of an unmanned mobile body control device 10g according to embodiment 8. The unmanned mobile body control device 10g includes an observation plan formulation unit 11g. The observation plan formulation unit 11g includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112, an image capture sensor image capture plan generation unit 113, a communicating unmanned mobile body movement plan generation unit 117, and an observation plan generation unit 114g.

[0139] The communication unmanned mobile unit movement plan generation unit 117 generates a movement plan for the communication unmanned mobile unit 40G, which performs terminal-to-terminal communication between the unmanned mobile unit 40g and the communication unmanned mobile unit 40G, and positions the communication unmanned mobile unit 40G at a position where the communication unmanned mobile unit 40G can communicate with a radio base station 30 on the communication path when transmitting mobile unit data including imaging data and imaging conditions obtained by the unmanned mobile unit 40g to a server 50, and where terminal-to-terminal communication is possible between the communication unmanned mobile unit 40G and the unmanned mobile unit 40g that cannot communicate with the radio base station 30.

[0140] The observation plan generation unit 114g generates an observation plan that includes the imaging target position information acquired from the imaging target position information storage unit 111, the movement plan acquired from the unmanned mobile unit movement plan generation unit 112, the imaging plan acquired from the imaging sensor imaging plan generation unit 113, and the movement plan for the communicating unmanned mobile unit 40G acquired from the communicating unmanned mobile unit movement plan generation unit 117. Note that the observation plan is not limited to the example described above, and may be configured not to include at least one of the imaging plan and the imaging target position information. In this way, the observation plan generation unit 114g generates an observation plan that includes at least the movement plan for the unmanned mobile unit 40g and the movement plan for the communicating unmanned mobile unit 40G.

[0141] Returning to the explanation of Fig. 29, the unmanned mobile body 40g that cannot communicate with the wireless base station 30 performs terminal-to-terminal communication to transmit mobile body data to the communicating unmanned mobile body 40G.

[0142] 31 is a diagram showing a configuration example of an unmanned moving body 40g according to embodiment 8. The unmanned moving body 40g includes a wireless terminal 41, a moving body control unit 42, an inter-terminal communication unit 49, an imaging sensor 43, and an imaging condition acquisition unit 44.

[0143] When the wireless terminal 41 can communicate with the wireless base station 30, it transmits the mobile data to the wireless base station 30. When the wireless terminal 41 cannot communicate with the wireless base station 30, it outputs the mobile data to the inter-terminal communication unit 49.

[0144] The inter-terminal communication unit 49 performs inter-terminal communication and transmits and receives inter-terminal communication data between the unmanned mobile unit 40g and the communicating unmanned mobile unit 40G, which can communicate with the wireless base station 30 on the communication path when transmitting mobile unit data to the server 50. The inter-terminal communication unit 49 transmits mobile unit data to the communicating unmanned mobile unit 40G when the wireless terminal 41 cannot communicate with the wireless base station 30. When the inter-terminal communication unit 49 receives an observation plan for its own terminal, i.e., the unmanned mobile unit 40g, from the communicating unmanned mobile unit 40G, it outputs the observation plan to the mobile unit control unit 42.

[0145] It should be noted that the unmanned mobile body 40g may be configured without the wireless terminal 41, provided that the unmanned mobile body 40G for communication is used in the three-dimensional remote sensing system 100g.

[0146] Returning to the explanation of Figure 29, the unmanned mobile body 40G for communication performs terminal-to-terminal communication between the unmanned mobile body 40g and the unmanned mobile body 40G for communication. The unmanned mobile body 40G for communication moves according to the movement plan of the unmanned mobile body 40G for communication, performs terminal-to-terminal communication, and transmits mobile body data received from the unmanned mobile body 40g that cannot communicate with the wireless base station 30 to the wireless base station 30.

[0147] 32 is a diagram showing a configuration example of a communicating unmanned mobile body 40G according to embodiment 8. The communicating unmanned mobile body 40G includes a wireless terminal 41, a communicating mobile body control unit 42G, and an inter-terminal communication unit 49.

[0148] The communication mobile unit control unit 42G controls the operation of the communication unmanned mobile unit 40G based on a movement plan for the communication unmanned mobile unit 40G, which positions the communication unmanned mobile unit 40G in a position where terminal-to-terminal communication between the unmanned mobile unit 40g and the communication unmanned mobile unit 40G is possible, so that the communication unmanned mobile unit 40G can communicate with a radio base station 30 on the communication path when transmitting mobile unit data including imaging data and imaging conditions obtained by the unmanned mobile unit 40g to a server 50, and so that terminal-to-terminal communication can be performed between the communication unmanned mobile unit 40G and the unmanned mobile unit 40g that cannot communicate with the radio base station 30.

[0149] The inter-terminal communication unit 49 performs terminal-to-terminal communication between the unmanned mobile unit 40g that cannot communicate with the wireless base station 30 and the communicating unmanned mobile unit 40G, and transmits and receives terminal-to-terminal communication data. Specifically, the inter-terminal communication unit 49 performs terminal-to-terminal communication between the unmanned mobile unit 40g that cannot communicate with the wireless base station 30 and the communicating unmanned mobile unit 40G, and receives mobile unit data from the unmanned mobile unit 40g that cannot communicate with the wireless base station 30. When the wireless terminal 41 receives an other terminal observation plan that is an observation plan for another terminal, for example, the unmanned mobile unit 40g that cannot communicate with the wireless base station 30, the inter-terminal communication unit 49 transmits the other terminal observation plan to the unmanned mobile unit 40g that cannot communicate with the wireless base station 30.

[0150] The wireless terminal 41 transmits the mobile data received by the inter-terminal communication unit 49 to the wireless base station 30. When the wireless terminal 41 receives the other-terminal observation plan, it outputs the other-terminal observation plan to the inter-terminal communication unit 49.

[0151] In the eighth embodiment, the three-dimensional remote sensing system 100g includes the communication unmanned vehicle 40G in addition to the unmanned vehicle 40g, but this is not limiting. For example, at least one of the multiple unmanned vehicles 40g may operate as the communication unmanned vehicle 40G.

[0152] As described above, according to this embodiment, when unmanned mobile unit 40g is unable to communicate with wireless base station 30, it performs terminal-to-terminal communication with communicating unmanned mobile unit 40G, causes communicating unmanned mobile unit 40G to transmit mobile unit data to wireless base station 30, and receives an observation plan addressed to its own terminal from communicating unmanned mobile unit 40G. This makes it possible for unmanned mobile unit 40g to transmit mobile unit data and receive an observation plan even when it is unable to communicate with wireless base station 30.

[0153] The three-dimensional remote sensing system 100g can be considered as a system for controlling the movement, i.e., flight, of the unmanned mobile body 40g using the communicating unmanned mobile body 40G. Figure 33 is a diagram showing an example configuration of an unmanned mobile body group control system 101g according to embodiment 8. The unmanned mobile body group control system 101g includes an unmanned mobile body control device 10g, a wireless base station 30, an unmanned mobile body 40g, and a communicating unmanned mobile body 40G.

[0154] The unmanned mobile body control device 10g generates a control plan that includes at least a movement plan for the unmanned mobile body 40g, and a movement plan for the communication unmanned mobile body 40G that positions the communication unmanned mobile body 40G in a position where the communication unmanned mobile body 40G, which performs terminal-to-terminal communication between the unmanned mobile body 40g and the communication unmanned mobile body 40G, can communicate with a wireless base station 30 on a communication path when the communication unmanned mobile body 40G transmits mobile body data obtained by the unmanned mobile body 40g to a server 50, and where terminal-to-terminal communication can be performed between the communication unmanned mobile body 40G and the unmanned mobile body 40g that cannot communicate with the wireless base station 30.

[0155] Unmanned mobile body 40g moves according to the movement plan of unmanned mobile body 40g, and when it can communicate with wireless base station 30, it transmits mobile body data to wireless base station 30, and when it cannot communicate with wireless base station 30, it performs terminal-to-terminal communication to transmit mobile body data to communicating unmanned mobile body 40G. Note that unmanned mobile body group control system 101g can also be applied to unmanned mobile bodies that do not have image sensors 43.

[0156] The unmanned mobile body 40G for communication moves according to the movement plan of the unmanned mobile body 40G for communication, performs terminal-to-terminal communication, and transmits mobile body data received from the unmanned mobile body 40g that cannot communicate with the wireless base station 30 to the wireless base station 30.

[0157] Embodiment 9 In the first embodiment, there are cases where appropriate imaging cannot be performed because the observation plan formulated in advance differs from the assumptions and does not match the local conditions of the imaging target, or because an error occurs in the assumed route due to the movement environment of the unmanned mobile body 40. There are also cases where the imaging conditions of the unmanned mobile body 40 deteriorate and it is not possible to restore the appropriate route. In the ninth embodiment, a case where an observation plan for the unmanned mobile body 40, specifically a movement plan for the unmanned mobile body 40, is corrected is described. Note that the explanation is given using the first embodiment as an example, but the ninth embodiment can also be applied to the second to eighth embodiments.

[0158] 34 is a diagram showing a configuration example of a three-dimensional remote sensing system 100h according to embodiment 9. The three-dimensional remote sensing system 100h includes an unmanned mobile object control device 10h, a wireless base station 30, an unmanned mobile object 40, and a server 50h. In embodiment 9, the imaging conditions used in embodiment 1 are the position of the imaging sensor 43 when it images the object 60, the angle of the imaging sensor 43 when it images the object 60, and the time when the imaging sensor 43 images the object 60.

[0159] The server 50h outputs the position, angle, and time acquisition status indicating the acquisition status of the position, angle, and time of the image sensor 43 included in the moving body data, and the three-dimensional data generation status indicating the extent to which the three-dimensional data has been generated.

[0160] 35 is a diagram showing an example of the configuration of a server 50h according to Embodiment 9. The server 50h includes a relative position / angle acquisition unit 51h and a three-dimensional data synthesis unit 52h.

[0161] Similar to the relative position / angle acquisition unit 51, the relative position / angle acquisition unit 51h outputs imaging data, object position, relative position, relative angle, and time, and further outputs a position / angle / time acquisition status indicating the acquisition status of the position, angle of the imaging sensor 43, and time included in the mobile object data. The relative position / angle acquisition unit 51h includes a data division unit 511, a relative position / angle calculation unit 512, and a position / angle / time acquisition status calculation unit 514. The position / angle / time acquisition status calculation unit 514 calculates and outputs the acquisition status of the position, angle, and time data output from the data division unit 511. Note that if the mobile object data includes acquisition status data, the position / angle / time acquisition status calculation unit 514 may use the acquisition status data included in the mobile object data.

[0162] The three-dimensional data synthesis unit 52h outputs three-dimensional data in the same manner as the three-dimensional data synthesis unit 52, and also outputs a three-dimensional data generation status indicating the extent to which the three-dimensional data has been generated. If there are defects in the three-dimensional data, such as missing data or distortion, the three-dimensional data generation status may include information indicating these defects.

[0163] The communication path for transmitting the position, angle, and time acquisition status and the three-dimensional data generation status from the server 50h to the unmanned mobile body control device 10h can be the same as the communication path for transmitting the observation plan from the unmanned mobile body control device 10h to the server 50h, but in the reverse direction.

[0164] Returning to the explanation of Fig. 34, the unmanned mobile body control device 10h corrects the movement plan of the unmanned mobile body 40 using the position, angle, and time acquisition status and the three-dimensional data generation status acquired from the server 50h.

[0165] 36 is a diagram showing an example of the configuration of an unmanned mobile body control device 10h according to Embodiment 9. The unmanned mobile body control device 10h includes an observation plan formulation unit 11h and an observation result correction review unit 12.

[0166] The observation result correction and review unit 12 receives from a server 50h that synthesizes 3D data representing the object 60 in three dimensions using mobile body data including imaging data obtained when the unmanned mobile body 40 captures an image of the object 60 using the imaging sensor 43, the position of the imaging sensor 43 when the imaging sensor 43 captured the object 60, the angle of the imaging sensor 43 when the imaging sensor 43 captured the object 60, and the time when the imaging sensor 43 captured the object 60. The server 50h receives the position, angle, and time acquisition status indicating the acquisition status of the imaging sensor 43, angle, and time, and 3D data generation status indicating the extent to which the 3D data has been generated, included in the mobile body data. The observation result correction and review unit 12 uses the received data to generate route correction data as information necessary to correct, i.e., update, the observation plan of the unmanned mobile body 40. The observation result correction and review unit 12 outputs the route correction data for correcting the movement route of the unmanned mobile body 40. The observation result correction examining unit 12 may use only one of the position, angle, and time acquisition status and the three-dimensional data generation status.

[0167] The observation plan formulation unit 11h includes an image capture target position information holding unit 111, an unmanned mobile body movement plan generation unit 112h, an image sensor image capture plan generation unit 113, and an observation plan generation unit 114. In the observation plan formulation unit 11h, the unmanned mobile body movement plan generation unit 112h uses path correction data to correct the movement plan for the unmanned mobile body 40. That is, the unmanned mobile body movement plan generation unit 112h generates a movement plan for the unmanned mobile body 40 using the path correction data in addition to the image capture target position information.

[0168] As described above, according to this embodiment, the server 50h outputs the position, angle, and time acquisition status and the three-dimensional data generation status, and the unmanned mobile body control device 10h uses the position, angle, and time acquisition status and the three-dimensional data generation status to correct the movement plan of the unmanned mobile body 40. As a result, even if the movement plan of the unmanned mobile body 40 becomes incompatible with the actual observation situation, the three-dimensional remote sensing system 100h can update the movement plan of the unmanned mobile body 40 to suit the situation, and can generate appropriate three-dimensional data.

[0169] Embodiment 10 In the ninth embodiment, the server 50h continuously transmits the position, angle, and time acquisition status and the three-dimensional data generation status to the unmanned mobile body control device 10h, resulting in a large amount of data being transmitted from the server 50h to the unmanned mobile body control device 10h. In the tenth embodiment, a case will be described in which the amount of data transmitted from the server to the unmanned mobile body control device is reduced while achieving the same effects as in the ninth embodiment. Note that the description will be given using the first embodiment as an example, but the tenth embodiment can also be applied to the second to eighth embodiments.

[0170] 37 is a diagram showing a configuration example of a three-dimensional remote sensing system 100i according to embodiment 10. The three-dimensional remote sensing system 100i includes an unmanned mobile object control device 10i, a wireless base station 30, an unmanned mobile object 40, and a server 50i. In embodiment 10, as in embodiment 9, the imaging conditions used in embodiment 1 are the position of the imaging sensor 43 when it images the object 60, the angle of the imaging sensor 43 when it images the object 60, and the time when the imaging sensor 43 images the object 60.

[0171] The server 50i uses the position, angle, and time acquisition status indicating the acquisition status of the position, angle, and time of the image sensor 43, included in the moving body data, and the three-dimensional data generation status indicating the extent to which the three-dimensional data has been generated, to generate route correction data for correcting the movement plan of the unmanned moving body 40 in the unmanned moving body control device 10i. The server 50i outputs the generated route correction data to the unmanned moving body control device 10i.

[0172] FIG. 38 is a diagram illustrating a configuration example of a server 50i according to the tenth embodiment. The server 50i includes a relative position / angle acquisition unit 51h, a three-dimensional data synthesis unit 52h, and an observation result correction / examination unit 53. In the tenth embodiment, the operation of the relative position / angle acquisition unit 51h and the three-dimensional data synthesis unit 52h is the same as that of the relative position / angle acquisition unit 51h and the three-dimensional data synthesis unit 52h according to the ninth embodiment. Furthermore, in the tenth embodiment, the operation of the observation result correction / examination unit 53 is the same as that of the observation result correction / examination unit 12 included in the unmanned mobile body control device 10h according to the ninth embodiment. The observation result correction / examination unit 53 uses the position, angle, and time acquisition status and the three-dimensional data generation status to output path correction data for correcting the movement path of the unmanned mobile body 40 in the unmanned mobile body control device 10i that generates a movement plan for the unmanned mobile body 40. It is assumed that the data volume of the path correction data is smaller than the data volume of the position, angle, and time acquisition status and the three-dimensional data generation status.

[0173] The communication path for transmitting the route correction data from the server 50i to the unmanned mobile body control device 10i can be the same as the communication path for transmitting the observation plan from the unmanned mobile body control device 10i to the server 50i, but in the reverse direction.

[0174] Returning to the explanation of Fig. 37, the unmanned mobile body control device 10i corrects the movement plan of the unmanned mobile body 40 using the route correction data. Although not shown in the figure, the unmanned mobile body control device 10i is the unmanned mobile body control device 10h of embodiment 9 from which the observation result correction review unit 12 has been removed. Therefore, detailed explanation will be omitted.

[0175] As described above, according to this embodiment, the server 50i generates and outputs path correction data using the position, angle, and time acquisition status and the 3D data generation status, and the unmanned mobile body control device 10i corrects the movement plan of the unmanned mobile body 40 using the path correction data. As a result, even if the movement plan of the unmanned mobile body 40 becomes incompatible with the actual observation situation, the 3D remote sensing system 100i can update the movement plan of the unmanned mobile body 40 to suit the situation and generate appropriate 3D data. Furthermore, by limiting the data transmitted from the server 50i to the unmanned mobile body control device 10i to the path correction data, the 3D remote sensing system 100i can reduce the amount of data transmitted from the server 50i to the unmanned mobile body control device 10i compared to embodiment 9. Therefore, the 3D remote sensing system 100i can be applied even in cases where real-time transmission of large amounts of data between the server 50i and the unmanned mobile body control device 10i is difficult.

[0176] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention. [Explanation of symbols]

[0177] 10, 10a, 10b, 10e, 10f, 10g, 10h, 10i unmanned mobile object control device, 11, 11a, 11b, 11e, 11f, 11g, 11h observation plan formulation unit, 12, 53 observation result correction examination unit, 20 ground network, 30 wireless base station, 40, 40-1 to 40-3, 40c, 40e, 40f, 40g unmanned mobile object, 40E unmanned mobile object for marker, 40F unmanned mobile object for positioning, 40G communication unmanned mobile object, 41 wireless terminal, 42 mobile object control unit, 42E mobile object control unit for marker, 42F mobile object control unit for positioning, 42G communication mobile object control unit, 43 imaging sensor, 44, 44c, 44e, 44f imaging condition acquisition unit, 45 Marker display unit, 46, 48 Position angle time acquisition unit, 47 Inter-terminal relative positioning unit, 49 Inter-terminal communication unit, 50, 50d, 50h, 50i Server, 51, 51d, 51h Relative position and relative angle acquisition unit, 52, 52h 3D data synthesis unit, 60 Object, 70 Positioning satellite, 80 Obstruction, 90, 94 Processing circuit, 91 Processor, 92 Memory, 93 Communication interface, 100, 100a, 100b, 100e, 100f, 100g, 100h, 100i 3D remote sensing system, 101e, 101f, 101g Unmanned mobile object group control system, 111 Imaged object position information storage unit, 112, 112a, 112b, 112h Unmanned mobile object movement plan generation unit, 113, 113a, 113b imaging sensor imaging plan generation unit, 114, 114e, 114f, 114g observation plan generation unit, 115 unmanned mobile object movement plan generation unit for marker, 116 unmanned mobile object movement plan generation unit for positioning, 117 unmanned mobile object movement plan generation unit for communication, 441, 481 GNSS receiver, 442, 482 inertial sensor, 443, 483 oscillator, 444, 444e, 444f, 484 position angle time calculation unit, 445, 513 viewpoint angle estimation unit, 511 data division unit, 512, 512d relative position relative angle calculation unit, 514 position angle time acquisition status calculation unit.

Claims

1. An unmanned mobile device control system that generates an observation plan including at least the movement plan of the unmanned mobile device, An unmanned mobile body that uses an imaging sensor to image an object based on the observation plan and generates imaging data and mobile body data including imaging conditions, A server that synthesizes three-dimensional data representing the object in three dimensions using the aforementioned moving object data, A marker-type unmanned mobile body that displays a marker on the aforementioned unmanned mobile body, Equipped with, The unmanned mobile device control device generates the observation plan, including the movement plan of the marker mobile device, such that the marker displayed by the marker mobile device is positioned at a location that is a feature point in the imaging data obtained when the unmanned mobile device images the target object. The unmanned mobile marker unit moves according to the movement plan of the unmanned mobile marker unit so that the marker is positioned at the location that constitutes the feature point, and displays the marker. The unmanned mobile unit further calculates the imaging conditions using the imaging data in which the marker is imaged along with the object. A three-dimensional remote sensing system characterized by the following:

2. An unmanned mobile device control device that generates an observation plan including at least a movement plan for an unmanned mobile device, An unmanned mobile body that uses an imaging sensor to image an object based on the observation plan and generates imaging data and mobile body data including imaging conditions, A server that synthesizes three-dimensional data representing the object in three dimensions using the aforementioned moving object data, Equipped with, The unmanned mobile device control system generates the observation plan such that the existing building is positioned as a characteristic point in the imaging data obtained when the unmanned mobile device images the target object. The unmanned mobile device further calculates the imaging conditions using the imaging data in which the existing building is imaged along with the target object. A three-dimensional remote sensing system characterized by the following:

3. The unmanned mobile device control system generates the observation plan for the unmanned mobile device to image the target object at different times. The unmanned mobile body images the target object at different times based on the observation plan, and generates mobile body data for each time the image was taken. The server synthesizes the three-dimensional data representing the object in three dimensions using the moving object data captured at each time point. The three-dimensional remote sensing system according to feature 1.

4. The unmanned mobile unit comprises multiple such units, The unmanned mobile device control system generates the observation plan for multiple unmanned mobile devices to image the target object. Multiple unmanned mobile units, based on the observation plan, image the target object and generate mobile unit data. The server synthesizes the three-dimensional data representing the object in three dimensions using the mobile object data received from a plurality of the unmanned mobile objects. The three-dimensional remote sensing system according to feature 1.

5. moreover, An unmanned mobile body for positioning performs terminal-to-terminal relative positioning to determine the terminal-to-terminal relative position between the aforementioned unmanned mobile body and the unmanned mobile body for positioning, Equipped with, The unmanned mobile device control device generates the observation plan which includes a movement plan for the unmanned mobile device for positioning, which positions the unmanned mobile device for positioning at a location where it can receive signals from positioning satellites to determine the position of the unmanned mobile device for positioning, and where relative positioning between terminals is possible between the unmanned mobile device for positioning and an unmanned mobile device that cannot receive signals from positioning satellites. The unmanned mobile positioning unit moves according to the movement plan of the unmanned mobile positioning unit, and performs terminal-to-terminal relative positioning with the unmanned mobile unit which cannot receive signals from the positioning satellite. If the unmanned mobile body is unable to receive signals from the positioning satellite, it moves according to the movement plan of the unmanned mobile body and performs terminal-to-terminal relative positioning with the positioning unmanned mobile body, and further calculates the imaging conditions using the terminal-to-terminal relative position. A three-dimensional remote sensing system according to any one of features 1 to 4.

6. moreover, The communication mobile unit performs terminal-to-terminal communication between the aforementioned unmanned mobile unit and the communication mobile unit, Equipped with, The unmanned mobile device control device generates the observation plan which includes a movement plan for the unmanned mobile device for communication, which positions the unmanned mobile device for communication at a location where it can communicate with a radio base station on the communication path when the unmanned mobile device for communication transmits the mobile device data to the server, and where terminal-to-terminal communication is possible between the unmanned mobile device for communication and an unmanned mobile device that cannot communicate with the radio base station. The unmanned mobile unit that cannot communicate with the aforementioned wireless base station performs terminal-to-terminal communication and transmits the mobile unit data to the aforementioned unmanned mobile unit for communication. The unmanned mobile communication unit moves according to its movement plan and performs terminal-to-terminal communication, and transmits the mobile unit data received from the unmanned mobile communication unit that cannot communicate with the wireless base station to the wireless base station. A three-dimensional remote sensing system according to any one of features 1 to 4.

7. The imaging conditions are defined as the position of the imaging sensor when it images the object, the angle of the imaging sensor when it images the object, and the time when the imaging sensor images the object. The server outputs a position-angle-time acquisition status indicating the acquisition status of the position of the image sensor, the angle of the image sensor, and the time included in the moving object data, and a 3D data generation status indicating the extent to which the 3D data has been generated. The unmanned mobile body control device corrects the movement plan of the unmanned mobile body using the position, angle, and time acquisition status and the 3D data generation status. A three-dimensional remote sensing system according to any one of features 1 to 4.

8. The imaging conditions are defined as the position of the imaging sensor when it images the object, the angle of the imaging sensor when it images the object, and the time when the imaging sensor images the object. The server uses the position, angle, and time acquisition status of the image sensor included in the mobile object data, and the 3D data generation status indicating the extent to which the 3D data has been generated, to output route correction data for correcting the movement plan of the unmanned mobile object in the unmanned mobile object control device. The unmanned mobile device control system corrects the movement plan of the unmanned mobile device using the path correction data. A three-dimensional remote sensing system according to any one of features 1 to 4.

9. An unmanned mobile body movement plan generation unit generates a movement plan for the unmanned mobile body using the image target location information, which is the location information of the object being imaged by the unmanned mobile body. An observation plan generation unit that generates an observation plan including at least the movement plan of the unmanned mobile body, A marker-use unmanned mobile body movement plan generation unit generates a movement plan for the marker-use unmanned mobile body so that the marker displayed by the marker-use unmanned mobile body is positioned at a location that is a characteristic point in the imaging data obtained when the unmanned mobile body images the target object. Equipped with, The observation plan generation unit generates an observation plan that includes at least the movement plan of the unmanned mobile body and the movement plan of the unmanned mobile body for the marker. An unmanned mobile device control system characterized by the following:

10. The unmanned mobile body movement plan generation unit generates the movement plan for the unmanned mobile body to image the target object at different times, moreover, An imaging sensor imaging plan generation unit generates an imaging plan for the unmanned mobile body to image the target object multiple times at different times, using the position information of the target object and the movement plan. Equipped with, The observation plan generation unit generates the observation plan, which includes at least the movement plan and the imaging plan. The unmanned mobile device control device according to feature 9.

11. The unmanned mobile body movement plan generation unit generates the movement plan for a plurality of the unmanned mobile bodies to image the target object, moreover, An imaging sensor imaging plan generation unit generates an imaging plan for multiple unmanned mobile bodies to image the target object using the position information of the target object and the movement plan. Equipped with, The observation plan generation unit generates the observation plan, which includes at least the movement plan and the imaging plan. The unmanned mobile device control device according to claim 9 or 10.

12. moreover, An observation result correction study unit receives from a server that synthesizes 3D data representing the object in three dimensions using imaging data obtained when the unmanned mobile body images the object using an image sensor, the position of the image sensor when the image sensor images the object, the angle of the image sensor when the image sensor images the object, and the time when the image sensor images the object, the position-angle-time acquisition status indicating the acquisition status of the position of the image sensor, the angle of the image sensor, and the time included in the mobile body data, and a 3D data generation status indicating the extent to which the 3D data has been generated, and outputs path correction data for correcting the movement path of the unmanned mobile body. Equipped with, The unmanned mobile body movement plan generation unit corrects the movement plan of the unmanned mobile body using the path correction data. The unmanned mobile device control device according to claim 9 or 10.

13. A mobile control unit that generates mobile control information for controlling the movement of the unmanned mobile body and imaging control information for controlling the imaging of an object by an imaging sensor, using an observation plan that includes at least the movement plan of the unmanned mobile body, Based on the aforementioned imaging control information, the imaging sensor captures an image of the target object, An imaging condition acquisition unit acquires imaging conditions when the imaging sensor captures the target object based on the moving object control information and the imaging control information, A wireless terminal that receives the observation plan and transmits mobile data including the object position indicating the location of the object, imaging data which is the imaging result of the imaging sensor, and imaging conditions, Equipped with, The aforementioned imaging condition acquisition unit is: A satellite positioning system receiver that receives signals from positioning satellites and outputs satellite positioning system data including position data by latitude and longitude and time data, An inertial sensor that detects three-dimensional inertial motion and outputs movement history data based on acceleration and angular velocity, An oscillator that outputs a clock that serves as a time reference, A viewpoint angle estimation unit uses the imaging data of the marker on the unmanned mobile marker unit, which displays a marker on the unmanned mobile marker unit together with the object, to estimate the relative position of the imaging sensor with respect to the marker and the relative angle of the imaging sensor with respect to the marker, and outputs the estimated relative position and relative angle as the estimated position and estimated angle. A position-angle-time calculation unit calculates, based on the satellite positioning system data, the movement history data, the clock, the mobile object control information, the imaging control information, the estimated position, and the estimated angle, the imaging conditions, which include the position of the imaging sensor when it images the object, the angle of the imaging sensor when it images the object, and the time when the imaging sensor images the object. An unmanned mobile vehicle characterized by having the following features.

14. A marker mobile unit control unit controls the operation of a marker mobile unit based on a movement plan for the marker mobile unit, which is generated such that the marker displayed by the marker mobile unit that displays the marker is positioned at a location that is a feature point in the imaging data obtained when the marker mobile unit images the object. A marker display unit that displays the aforementioned marker, An unmanned mobile marker characterized by being equipped with the following features.

15. An unmanned mobile device control device that generates a control plan including at least a movement plan for an unmanned mobile device, and a movement plan for a marker unmanned mobile device such that the marker displayed by the marker unmanned mobile device is positioned at a location that is a characteristic point in the imaging data obtained when the unmanned mobile device images an object, The unmanned mobile body for markers moves according to the movement plan of the unmanned mobile body for markers so that the marker is positioned at the location of the feature point, and displays the marker on the unmanned mobile body, An unmanned mobile body that, based on the control plan, uses an image sensor to image the target object, calculates imaging conditions using the image data in which the marker is imaged along with the target object, and generates mobile body data including the image data and the imaging conditions, An unmanned mobile unit swarm control system characterized by comprising the following features.

16. A control circuit for controlling an unmanned mobile device control system, Using the image target location information, which is the location information of the object being imaged by the unmanned mobile body, a movement plan for the unmanned mobile body is generated. The movement plan for the marker-using mobile device is generated such that the marker displayed by the marker-using mobile device is positioned at a location that is a characteristic point in the imaging data obtained when the unmanned mobile device images the target object. An observation plan is generated that includes at least the movement plan of the unmanned mobile body and the movement plan of the unmanned mobile body for the marker. A control circuit characterized by causing the unmanned mobile device control device to perform the following.

17. A control circuit for controlling an unmanned mobile vehicle, We received an observation plan that included at least the movement plan of an unmanned mobile object. Using the observation plan, mobile body control information for controlling the movement of the unmanned mobile body and imaging control information for controlling the imaging of the target object by the imaging sensor are generated. The imaging sensor captures the target object based on the imaging control information. Based on the aforementioned mobile body control information and the aforementioned imaging control information, the imaging conditions when the imaging sensor images the target object are obtained. The object position indicating the location of the object, the imaging data which is the imaging result of the imaging sensor, and the moving object data including the imaging conditions are transmitted. The unmanned mobile vehicle is to perform this action. In acquiring the aforementioned imaging conditions, It receives signals from positioning satellites and outputs satellite positioning system data, including position data by latitude and longitude, and time data. It detects three-dimensional inertial motion and outputs movement history data based on acceleration and angular velocity. Outputs a clock that serves as the time reference. Using the image data in which the marker of the unmanned mobile marker, which displays a marker on the unmanned mobile marker together with the object, is imaged, the relative position of the image sensor with respect to the marker and the relative angle of the image sensor with respect to the marker are estimated, and the estimated relative position and relative angle are output as the estimated position and estimated angle. Based on the satellite positioning system data, the movement history data, the clock, the mobile object control information, the imaging control information, the estimated position, and the estimated angle, the imaging conditions are calculated as follows: the position of the imaging sensor when the imaging sensor images the object, the angle of the imaging sensor when the imaging sensor images the object, and the time when the imaging sensor images the object. A control circuit characterized by having the unmanned mobile body perform the operation.

18. A storage medium in which a program for controlling an unmanned mobile device is stored, The aforementioned program, Using the image target location information, which is the location information of the object being imaged by the unmanned mobile body, a movement plan for the unmanned mobile body is generated. The movement plan for the marker-using mobile device is generated such that the marker displayed by the marker-using mobile device is positioned at a location that is a characteristic point in the imaging data obtained when the unmanned mobile device images the target object. An observation plan is generated that includes at least the movement plan of the unmanned mobile body and the movement plan of the unmanned mobile body for the marker. A storage medium characterized by causing the unmanned mobile device control system to perform the following action.

19. A storage medium in which a program for controlling an unmanned mobile object is stored, The aforementioned program, We received an observation plan that included at least the movement plan of an unmanned mobile object. Using the observation plan, mobile body control information for controlling the movement of the unmanned mobile body and imaging control information for controlling the imaging of the target object by the imaging sensor are generated. The imaging sensor captures the target object based on the imaging control information. Based on the aforementioned mobile body control information and the aforementioned imaging control information, the imaging conditions when the imaging sensor images the target object are obtained. The object position indicating the location of the object, the imaging data which is the imaging result of the imaging sensor, and the moving object data including the imaging conditions are transmitted. The unmanned mobile vehicle is to perform this action. In acquiring the aforementioned imaging conditions, It receives signals from positioning satellites and outputs satellite positioning system data, including position data by latitude and longitude, and time data. It detects three-dimensional inertial motion and outputs movement history data based on acceleration and angular velocity. Outputs a clock that serves as the time reference. Using the image data in which the marker of the unmanned mobile marker, which displays a marker on the unmanned mobile marker together with the object, is imaged, the relative position of the image sensor with respect to the marker and the relative angle of the image sensor with respect to the marker are estimated, and the estimated relative position and relative angle are output as the estimated position and estimated angle. Based on the satellite positioning system data, the movement history data, the clock, the mobile object control information, the imaging control information, the estimated position, and the estimated angle, the imaging conditions are calculated as follows: the position of the imaging sensor when the imaging sensor images the object, the angle of the imaging sensor when the imaging sensor images the object, and the time when the imaging sensor images the object. A storage medium characterized by having the unmanned mobile body perform the action.

20. The unmanned mobile device control system performs a first step of generating an observation plan which includes at least a movement plan for the unmanned mobile device, The second step is for the unmanned mobile marker to display a marker on the unmanned mobile marker, The third step involves the unmanned mobile unit imaging an object using an imaging sensor based on the observation plan, The fourth step involves the unmanned mobile body generating imaging data and mobile body data including imaging conditions, A fifth step in which the server synthesizes three-dimensional data representing the object in three dimensions using the moving object data, Includes, In the first step, the unmanned mobile device control device generates the observation plan, including the movement plan of the marker mobile device, such that the marker displayed by the marker mobile device is positioned at a location that is a feature point in the imaging data obtained by the unmanned mobile device imaging the object. In the second step, the unmanned mobile marker unit moves according to the movement plan of the unmanned mobile marker unit so that the marker is positioned at the location that constitutes the feature point, and displays the marker. In the fourth step, the unmanned mobile body further calculates the imaging conditions using the imaging data in which the marker is imaged together with the object. A method for operating a three-dimensional remote sensing system, characterized by the following:

21. The first step involves the unmanned mobile body movement plan generation unit generating a movement plan for the unmanned mobile body using the image target location information, which is the location information of the object being imaged by the unmanned mobile body. The observation plan generation unit generates an observation plan that includes at least the movement plan of the unmanned mobile body, A third step is for the unmanned mobile marker movement plan generation unit to generate a movement plan for the unmanned mobile marker such that the marker displayed by the unmanned mobile marker is positioned at a location that is a feature point in the imaging data obtained when the unmanned mobile marker images the target object. Includes, In the second step described above, the observation plan generation unit generates an observation plan that includes at least the movement plan of the unmanned mobile body and the movement plan of the unmanned mobile body for the marker. An operating method for an unmanned mobile device control system, characterized by the following:

22. A wireless terminal receives an observation plan which includes at least the movement plan of an unmanned mobile object, A second step involves the mobile unit control unit generating mobile unit control information for controlling the movement of the unmanned mobile unit and imaging control information for controlling the imaging of an object by the imaging sensor, using the observation plan. The imaging sensor performs a third step of imaging the target object based on the imaging control information, A fourth step is for the imaging condition acquisition unit to acquire the imaging conditions when the imaging sensor images the target object, based on the moving object control information and the imaging control information. A fifth step in which the wireless terminal transmits object position data including the object position indicating the location of the object, imaging data which is the imaging result of the imaging sensor, and imaging conditions, Includes, The fourth step described above is: A satellite positioning system receiver receives signals from positioning satellites and outputs satellite positioning system data including position data by latitude and longitude and time data; The inertial sensor detects three-dimensional inertial motion and outputs motion history data based on acceleration and angular velocity in an inertial step, The oscillator performs an oscillation step in which it outputs a time-referenced clock, A viewpoint angle estimation step in which the viewpoint angle estimation unit uses the imaging data in which the marker of the marker-using unmanned mobile body that displays the marker on the object together with the unmanned mobile body is imaged to estimate the relative position of the imaging sensor with respect to the marker and the relative angle of the imaging sensor with respect to the marker, and outputs the estimated relative position and relative angle as estimated position and estimated angle, Position angle time calculation step, in which the position angle time calculation unit calculates the position of the image sensor when the image sensor captures the target object, the angle of the image sensor when the image sensor captures the target object, and the time when the image sensor captures the target object, based on the satellite positioning system data, the movement history data, the clock, the mobile object control information, the image capture control information, the estimated position, and the estimated angle, as the imaging conditions, An unmanned mobile body operation method characterized by including the following.

Images